Maternal and Infant Health: HHS Should Strengthen Processes for Measuring Program Performance
Fast Facts
The U.S. has the highest rates of maternal and infant death of developed countries. The Healthy Start program tries to address this with 3 goals:
Reduce infant death rates
Improve maternal health
Reduce racial and ethnic disparities in health outcomes
The Department of Health and Human Services collects data to measure the program's progress on reducing infant deaths and improving maternal health. HHS said it plans to analyze the data by race and ethnicity. But it's unclear how reliable HHS's data is.
Also, HHS could better coordinate to measure Healthy Start and 2 other HHS programs with similar goals.
We recommended addressing these issues.
Highlights
What GAO Found
The U.S. has the highest rates of maternal and infant death of developed, high-income countries. The Healthy Start program, administered by the Department of Health and Human Services (HHS), has three goals: reduce rates of infant death, improve maternal health outcomes, and reduce racial and ethnic disparities.
HHS funded 101 Healthy Start grantees—e.g., health departments and not-for-profit organizations—in the 2019-2024 grant period. In 2022, grantees served nearly 85,000 participants, including pregnant women, partners, and children. Slightly more than half of participants were Black or African American. Grantees are expected to provide one-on-one consultation and health and parenting education classes, as well as connect participants with medical services.
Percentage of Healthy Start Participants by Race in 2022
Note: “Other” includes Asian, Native Hawaiian or other Pacific Islander, and multiple races.
HHS uses performance data to assess progress toward the first two of Healthy Start's three goals. It has plans to begin stratifying these data by race and ethnicity in 2024 to assess progress toward the third goal: reducing racial and ethnic disparities. HHS is also conducting an evaluation, to be completed in 2025, to assess program progress toward all three goals.
Healthy Start data to measure program performance—known as performance measures—align with most GAO-identified key attributes of effective measures. However, the measures do not align with two attributes related to clarity and reliability. For example, HHS's updates to the measures during the 2019-2024 grant period made it challenging for grantees to collect consistent and therefore reliable data. This occurred because HHS lacks a documented process to review its performance measures before implementation. Such a process would help ensure HHS is using measures that are clear and allow for the collection of reliable data.
Also, HHS officials said they try to align performance measures across Healthy Start and two related programs. However, HHS lacks a documented process to coordinate the selection of measures used across the programs, and officials could not provide examples of coordination. Implementing a documented process could help ensure that HHS is using the most appropriate measures. Ultimately, this could help HHS gather the best evidence across the programs as they pursue the shared goal of improving health outcomes for mothers and infants.
Why GAO Did This Study
The Healthy Start program awards funding to organizations in areas with high rates of infant death and other adverse maternal and infant health outcomes. In fiscal year 2022, the Healthy Start program received $131 million in appropriations.
The CARES Act included a provision for GAO to review and assess Healthy Start. This report: describes Healthy Start grantees and participants served, examines HHS's efforts to assess progress towards Healthy Start goals, examines the extent to which Healthy Start performance measures meet key attributes of effective measures, and examines HHS' efforts to align Healthy Start performance measures with those of related HHS programs.
GAO analyzed Healthy Start and other HHS program documentation and data for the most recent grant period (April 2019–March 2024) and the upcoming period (beginning April 2024), including performance measure information. GAO also interviewed HHS officials and six Healthy Start grantees, selected to vary by factors such as geography and years as a grantee.
Recommendations
GAO is making two recommendations. HHS should implement documented processes to 1) review Healthy Start performance measures to ensure they are clear and allow for the collection of reliable data, and 2) coordinate the selection of performance measures across Healthy Start and related programs. HHS partially agreed with the first recommendation and agreed with the second. GAO maintains the recommendations are warranted, as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Health Resources and Services Administration | The Administrator of HRSA should implement a documented process to review Healthy Start performance measures (including their definitions, associated data collection forms, and data dictionary), before a grant period begins, to ensure they are clear and allow for the collection of reliable data. (Recommendation 1) |
HHS partially agreed with this recommendation. In September 2024, HHS provided a standard operating procedure (SOP) HRSA created in August 2024 to be used for reviewing Healthy Start performance measures. This SOP outlines considerations and steps that program staff should take to determine whether to update the program's performance measure definitions, associated data collection forms, and data dictionary. It states that implementation of the SOP should start 2 years prior to the next grant period; however, the SOP lacks details about the timing needed to conduct the steps within the SOP to ensure that all SOP steps are completed before a grant period begins. The SOP also lacks details on how the process ensures that the measures (including their definitions, associated data collection forms, and data dictionary) are clear to grantees. GAO will continue to monitor HRSA's efforts towards implementing this recommendation to ensure the agency's actions result in the establishment of clear measures before the next grant period begins. Therefore, GAO considers this recommendation open.
|
| Health Resources and Services Administration | The Administrator of HRSA should implement a documented process for program officials to coordinate the selection of performance measures across its related programs—Healthy Start; the Maternal, Infant, and Early Childhood Home Visiting; and the Title V Maternal and Child Health Services Block Grant. (Recommendation 2) |
HHS concurred with this recommendation. In September 2024, HHS provided a HRSA memo from April 2024 that detailed steps the agency planned to implement to review existing performance measures across its related programs. The memo described which staff were to be involved, what activities they would complete, and the associated time frames. For example, one step was to convene a workgroup to review existing performance measures. According to HHS, the workgroup was formed and developed options to align measures but had not finalized any decisions as of September 2024. GAO will continue to monitor HRSA's progress toward implementing this recommendation, including reviewing the workgroup's decisions. Further, HRSA's steps, as described in the memo, are one-time only actions. Given that performance measures may change over time, a process to ensure that coordination occurs on a recurring basis is needed, per GAO's recommendation. Therefore, GAO considers this recommendation open.
|