President's Emergency Plan for AIDS Relief: Shift toward Partner- Country Treatment Programs Will Require Better Information on Results
Highlights
What GAO Found
The Department of State's (State) Office of the U.S. Global AIDS Coordinator (OGAC) has reported on President's Emergency Plan for AIDS Relief (PEPFAR) treatment program results primarily in terms of (1) numbers of people on treatment directly supported by PEPFAR, (2) percentages of eligible people receiving treatment, and (3) percentages of people alive and on treatment 12 months after starting treatment. However, these indicators do not reflect some key PEPFAR results. First, although the number of people on treatment directly supported by PEPFAR grew from about 1.7 million to 5.1 million in fiscal years 2008 through 2012, this indicator alone does not provide complete information needed for assessing PEPFAR's contributions to partner countries' treatment programs. Second, although 10 PEPFAR country teams reported that percentages of people alive and on treatment after 12 months exceeded 80 percent, data for this indicator are not always complete and have other limitations. To improve these data, according to OGAC officials, OGAC clarified its guidance and conducted data quality assessments. However, OGAC has not yet established a common set of indicators to monitor the results of PEPFAR's efforts to improve the quality of treatment programs.
As PEPFAR partner countries assume greater responsibility for managing their treatment programs, fully functioning monitoring and evaluation (M&E) systems are critical for tracking results and ensuring treatment program effectiveness. PEPFAR country teams assist partner countries in carrying out their M&E responsibilities by providing staff, training, technical assistance, and other support. With this assistance, partner countries have made some progress in expanding and upgrading these M&E systems. Nevertheless, partner countries' M&E systems often are unable to produce complete and timely data, thus limiting the usefulness of these data for patient, clinic, or program management. OGAC has not yet established minimum standards for partner countries' M&E systems, particularly relating to data completeness and timeliness, in order for PEPFAR country teams to assess those systems' readiness for use in treatment program management and results reporting.
Why GAO Did This Study
PEPFAR, first authorized in 2003, has supported significant advances in HIV/AIDS prevention, treatment, and care in more than 30 countries. In reauthorizing the program in 2008, Congress directed OGAC to continue to expand the number of people receiving care and treatment through PEPFAR while also making it a major policy goal to help partner countries develop independent, sustainable HIV programs. As a result, PEPFAR began shifting efforts from directly providing treatment services toward support for treatment programs managed by partner countries. GAO was asked to review PEPFAR treatment programs. GAO examined (1) PEPFAR treatment program results and how OGAC measures them and (2) PEPFAR assistance to improve partner countries' M&E systems. GAO reviewed PEPFAR plans, performance reports, and guidance and interviewed officials from OGAC, the Centers for Disease Control and Prevention (CDC), and the U.S. Agency for International Development (USAID). GAO also synthesized findings of treatment program studies and conducted fieldwork in three countries.
Recommendations
The Secretary of State should direct OGAC to (1) develop a method that better accounts for PEPFAR's contributions to partner-country treatment programs, (2) establish a common set of indicators to measure the results of treatment program quality improvement efforts, and (3) establish a set of minimum standards for data generated by partner countries' M&E systems. Commenting jointly with CDC and USAID, State generally agreed with the report's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of State | To ensure the outcomes and quality of treatment programs supported by PEPFAR, in collaboration with PEPFAR implementing agencies, the Secretary of State should direct the U.S. Global AIDS Coordinator to develop a method that better accounts for PEPFAR's contributions to partner-country treatment programs. |
State generally agreed with this recommendation. According to an official from the Office of the Global AIDS Coordinator (OGAC), State took the actions outlined below at least partially in response to GAO's recommendation. The official added that even in those instances where some of the actions were already in process, the GAO recommendation gave OGAC considerably more incentive and energy. According to information provided by OGAC in April 2017, PEPFAR has defined three categories of contributions to country treatment programs: (1) direct service delivery, which entails directly interacting with patients or beneficiaries by providing key staff and/or commodities and support to improve the quality of services through site visits as often as deemed necessary by the partner and country team; (2) technical assistance/service delivery improvement, which entails a minimum of one site visit per fiscal year; and (3) central support, meaning annual site visits in areas where PEPFAR is providing solely financial support at the national, regional or district level. PEPFAR officials stated that PEPFAR developed these three categories after completing a second year of quarterly site-level monitoring, which provided data that demonstrate important differences in patient outcomes and site performance.
|
| Department of State | To ensure the outcomes and quality of treatment programs supported by PEPFAR, in collaboration with PEPFAR implementing agencies, the Secretary of State should direct the U.S. Global AIDS Coordinator to establish a common set of indicators to measure the results of treatment program quality improvement efforts. |
State generally agreed with this recommendation. According to an OGAC official, State took the actions outlined below at least partially in response to GAO's recommendation. The official added that even in those instances where some of the actions were already in process, the GAO recommendation gave OGAC considerably more incentive and energy. According to information provided by OGAC in 2015, PEPFAR introduced new indicators as part of a larger FY2014 monitoring, evaluation, and reporting strategy. The new indicators include (1) patients receiving a viral load result within the previous twelve months and (2) the proportion of viral load tests with an undetectable viral load count. A viral load test measures how much human immunodeficiency virus (HIV) is in the blood. Both of these indicators go beyond a retention rate metric introduced earlier (patients remaining alive and in treatment one year after starting treatment). In addition to these new metrics, PEPFAR began a more encompassing strategy to link relevant indicators into a cascade of service delivery-from initial counseling and testing, through entry into the care system, and ultimately to viral load measures-to monitor the entirety of HIV service delivery. OGAC provided updated information in April 2017, stating that PEPFAR streamlined indicators to incorporate geographic and demographic distinctions, and that the revised indicators provide a framework for monitoring key aspects of the UNAIDS 90-90-90 global goals: 90 percent of people living with HIV know their HIV status, 90 percent of those who know their status are on treatment, and 90 percent of those on treatment have viral suppression. PEPFAR's 2017 Annual Report to Congress clarified that these indicators are also meant to simplify data collection and reduce required data points, align gender and age disaggregation with global practices, strengthen alignment of indicators with multilateral partners, and increase reporting frequency to align with and support national and multilateral procedures. PEPFAR officials in 2017 stated that, in addition to core clinical indicators, PEPFAR will also track progress toward prevention goals and activities related to treating tuberculosis, one of the leading causes of death among people living with HIV.
|
| Department of State | To ensure the outcomes and quality of treatment programs supported by PEPFAR, in collaboration with PEPFAR implementing agencies, the Secretary of State should direct the U.S. Global AIDS Coordinator to establish a set of minimum standards for data generated by partner countries' M&E systems, to enable PEPFAR country teams to assess those systems' readiness for use in treatment program management and reporting. |
State generally agreed with this recommendation. According to an OGAC official, State took the actions outlined below at least partially in response to GAO's recommendation. The official added that even in those instances where some of the actions were already in process, the GAO recommendation gave OGAC considerably more incentive and energy. In commenting on the report, State noted that PEPFAR does not independently create data standards for partner country reporting systems, but does support global efforts to define relevant standards that are developed in conjunction with a multilateral consensus process. State noted that it was working with partner countries on data systems and reporting timeframes to address timeliness; that it supported the World Health Organization?s efforts to develop consensus for and publish international standards for data exchange; and that it was working with the Global Fund to support the implementation of the District Health Information Software (DHIS) system, used by many countries to manage aggregate data for national, PEPFAR, Global Fund, and other donor reporting. According to information provided by OGAC in 2015, countries were increasingly choosing this system as the foundation for their national health data systems. Moreover, the transition to this system had gained considerable momentum, since PEPFAR had used this technology as the platform for its new Data Accountability, Transparency, and Impact system, introduced in 2015, which supports the reporting of targets and results from PEPFAR implementing partners for each service delivery site included in each partner?s portfolio. OGAC officials stated that PEPFAR selected the DHIS system since many countries were using it and the code is open-source. In addition, a large community of developers and users had evolved over the last few years, and everything PEPFAR was developing was available to country partners for use in their own systems. The officials stated that PEPFAR was actively engaged with these same country partners to integrate systems to the extent possible to support a single - rather than parallel - approach to data management. OGAC clarified in April 2017 that country-specific implementation of a DHIS or comparable system would minimize data entry burden and improve overall data quality, but would not replace PEPFAR's Data Accountability, Transparency, and Impact system for monitoring partner performance. PEPFAR's 2017 Country/Regional Operation Plan Guidance sets as a goal the availability and use of high quality data and states that PEPFAR is working with host country governments and other stakeholders to improve the frequency and quality of survey and surveillance data, with impact assessments expected to be completed in 11 countries over the next two years. In addition, PEPFAR's 2017 Annual Report to Congress stated that PEPFAR aims to increase reporting frequency to align with and support national and multilateral procedures, which could help address data timeliness issues.
|