This is the accessible text file for GAO report number GAO-13-460 
entitled 'President's Emergency Plan For Aids Relief: Shift toward 
Partner-Country Treatment Programs Will Require Better Information on 
Results' which was released on May 13, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

April 2013: 

President's Emergency Plan For Aids Relief: 

Shift toward Partner-Country Treatment Programs Will Require Better 
Information on Results: 

GAO-13-460: 

GAO Highlights: 

Highlights of GAO-13-460, a report to congressional requesters. 

Why GAO Did This Study: 

PEPFAR, first authorized in 2003, has supported significant advances 
in HIV/AIDS prevention, treatment, and care in more than 30 countries. 
In reauthorizing the program in 2008, Congress directed OGAC to 
continue to expand the number of people receiving care and treatment 
through PEPFAR while also making it a major policy goal to help 
partner countries develop independent, sustainable HIV programs. As a 
result, PEPFAR began shifting efforts from directly providing 
treatment services toward support for treatment programs managed by 
partner countries. GAO was asked to review PEPFAR treatment programs. 
GAO examined (1) PEPFAR treatment program results and how OGAC 
measures them and (2) PEPFAR assistance to improve partner countries’ 
M&E systems. GAO reviewed PEPFAR plans, performance reports, and 
guidance and interviewed officials from OGAC, the Centers for Disease 
Control and Prevention (CDC), and the U.S. Agency for International 
Development (USAID). GAO also synthesized findings of treatment 
program studies and conducted fieldwork in three countries. 

What GAO Found: 

The Department of State’s (State) Office of the U.S. Global AIDS 
Coordinator (OGAC) has reported on President’s Emergency Plan for AIDS 
Relief (PEPFAR) treatment program results primarily in terms of (1) 
numbers of people on treatment directly supported by PEPFAR, (2) 
percentages of eligible people receiving treatment, and (3) 
percentages of people alive and on treatment 12 months after starting 
treatment (see table for recently reported results). However, these 
indicators do not reflect some key PEPFAR results. First, although the 
number of people on treatment directly supported by PEPFAR grew from 
about 1.7 million to 5.1 million in fiscal years 2008 through 2012, 
this indicator alone does not provide complete information needed for 
assessing PEPFAR’s contributions to partner countries’ treatment 
programs. Second, although 10 PEPFAR country teams reported that 
percentages of people alive and on treatment after 12 months exceeded 
80 percent, data for this indicator are not always complete and have 
other limitations. To improve these data, according to OGAC officials, 
OGAC clarified its guidance and conducted data quality assessments. 
However, OGAC has not yet established a common set of indicators to 
monitor the results of PEPFAR’s efforts to improve the quality of 
treatment programs. 

Table: PEPFAR Treatment Indicators: 

Indicator: Number of people currently on treatment directly supported 
by PEPFAR; 
Source of data: PEPFAR direct treatment programs; 
Target: 5 million (fiscal year 2012); 
Reported results: 5.1 million (as of fiscal year 2012). 

Indicator: Percentage of eligible people receiving treatment; 
Source of data: Partner country treatment programs; 
Target: 80 percent; 
Reported results: 8 of 23 partner countries reported rates at or above 
80 percent (as of 2011). 

Indicator: Percentage of people alive and on treatment 12 months after 
starting treatment; 
Source of data: PEPFAR direct treatment programs; 
Target: No target; 
Reported results: 10 of 23 partner countries reported rates at or 
above 80 percent (as of fiscal year 2012). 

Source: GAO synthesis of guidance and data from OGAC and data from the 
Joint United Nations Programme on HIV/AIDS. 

Notes: In fiscal year 2012, 23 PEPFAR country teams providing direct 
support for treatment services were required to report to OGAC on 
these indicators. All 23 country teams provided data on number of 
people currently on treatment directly supported by PEPFAR and 
percentage of eligible people receiving treatment. Twenty country 
teams provided data on percentage of people alive and on treatment 12 
months after starting treatment; 3 countries reported that data were 
not available for this indicator. 

[End of table] 

As PEPFAR partner countries assume greater responsibility for managing 
their treatment programs, fully functioning monitoring and evaluation 
(M&E) systems are critical for tracking results and ensuring treatment 
program effectiveness. PEPFAR country teams assist partner countries 
in carrying out their M&E responsibilities by providing staff, 
training, technical assistance, and other support. With this 
assistance, partner countries have made some progress in expanding and 
upgrading these M&E systems. Nevertheless, partner countries’ M&E 
systems often are unable to produce complete and timely data, thus 
limiting the usefulness of these data for patient, clinic, or program 
management. OGAC has not yet established minimum standards for partner 
countries’ M&E systems, particularly relating to data completeness and 
timeliness, in order for PEPFAR country teams to assess those systems’ 
readiness for use in treatment program management and results 
reporting. 

What GAO Recommends: 

The Secretary of State should direct OGAC to (1) develop a method that 
better accounts for PEPFAR’s contributions to partner-country 
treatment programs, (2) establish a common set of indicators to 
measure the results of treatment program quality improvement efforts, 
and (3) establish a set of minimum standards for data generated by 
partner countries’ M&E systems. Commenting jointly with CDC and USAID, 
State generally agreed with the report’s recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-13-460]. For more 
information, contact David Gootnick at (202) 512-3149 or 
gootnickd@gao.gov, or Marcia Crosse at (202) 512-7114 or 
crossem@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Data Indicate Progress in Achieving Treatment Program Results, but 
Some Indicators Have Limitations: 

PEPFAR Supports Countries' M&E Systems, but Weaknesses Limit the 
Timeliness and Completeness of Treatment Results Data: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Criteria for Assessing PEPFAR Direct Support: 

Appendix III: Summary of Studies: 

Appendix IV: Comments from the Department of State: 

Appendix V: GAO Contacts and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Key PEPFAR Treatment Indicators: 

Table 2: Numbers of People on Treatment Directly Supported by PEPFAR 
and Numbers of PEPFAR Countries Reporting on This Indicator, Fiscal 
Years 2004 through 2012: 

Table 3: Estimated PEPFAR Partner-Country Treatment Coverage Rates, 2011: 

Table 4: Reported PEPFAR Treatment Retention Rates, Fiscal Year 2012: 

Table 5: Checklist to Determine Whether to Count Service Delivery 
Support as PEPFAR Direct: 

Figure: 

Figure 1: Number of People on Treatment Directly Supported by PEPFAR, 
Fiscal Years 2004-2012, and Total Number of People on Treatment in 
Low- and Middle-Income Countries, Calendar Years 2004-2011: 

Abbreviations: 

2008 Leadership Act: Tom Lantos and Henry J. Hyde United States Global 
Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization 
Act of 2008: 

ARV: antiretroviral: 

CD4: cluster of differentiation antigen 4: 

CDC: Centers for Disease Control and Prevention: 

Global Fund: Global Fund to Fight AIDS, Tuberculosis and Malaria: 

HHS: Department of Health and Human Services: 

M&E: monitoring and evaluation: 

OGAC: Office of the U.S. Global AIDS Coordinator: 

PEPFAR: President's Emergency Plan for AIDS Relief: 

State: Department of State: 

UNAIDS: Joint United Nations Programme on HIV/AIDS: 

USAID: U.S. Agency for International Development: 

WHO: World Health Organization: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

April 12, 2013: 

Congressional Requesters: 

As the President's Emergency Plan for AIDS Relief (PEPFAR) nears the 
end of its second 5-year authorization,[Footnote 1] it has made 
significant contributions to international treatment targets for 
HIV/AIDS.[Footnote 2] The Department of State's (State) Office of the 
U.S. Global AIDS Coordinator (OGAC) reported that, as of September 30, 
2012, the U.S. government, through PEPFAR's multibillion dollar 
investments in partner countries' treatment and care programs, had 
directly supported treatment for more than 5.1 million people--more 
than half of all people on treatment for HIV/AIDS in low-and middle-
income countries. In part because of these investments, more than half 
of eligible people in low-and middle-income countries (54 percent) 
receive treatment, and the estimated number of AIDS-related deaths 
worldwide (1.7 million) declined in 2011, according to data reported 
by the United Nations Joint Programme on HIV/AIDS (UNAIDS). 

Nevertheless, important challenges remain as PEPFAR continues to help 
build partner countries' capacity to manage their treatment programs. 
First, the estimated number of new HIV infections worldwide each year 
(2.5 million in 2011) continues to exceed the estimated increase in 
the number of people on treatment (about 1.4 million from 2010 to 
2011), and about 6.8 million people eligible for treatment have not 
yet been treated.[Footnote 3] In addition, in passing the Tom Lantos 
and Henry J. Hyde United States Global Leadership Against HIV/AIDS, 
Tuberculosis, and Malaria Reauthorization Act of 2008 (2008 Leadership 
Act),[Footnote 4] Congress directed OGAC to continue to expand the 
number of people receiving PEPFAR HIV care and treatment services 
while also making it a major policy goal to build partner-country 
capacity to deliver services and promote a transition toward greater 
sustainability of country-owned HIV/AIDS programs. In December 2011, 
the President announced an increase in PEPFAR's target for the number 
of people receiving treatment directly supported by PEPFAR--from 4 
million to 6 million by the end of fiscal year 2013. 

OGAC sets overall PEPFAR policy and strategies. OGAC also coordinates 
PEPFAR programs and activities, allocating funds to PEPFAR 
implementing agencies, particularly the Centers for Disease Control 
and Prevention (CDC) of the Department of Health and Human Services 
(HHS) and the U.S. Agency for International Development (USAID). OGAC 
is leading PEPFAR's transition from an early focus on directly 
providing treatment services as part of an emergency response, to 
building partner countries' capacity to manage treatment programs. 
During this transition, ensuring HIV care and treatment program 
quality remains critical to achieving patient and public health 
outcomes, even as annual allocations to PEPFAR remain unchanged or 
decline.[Footnote 5] 

You asked us to review HIV/AIDS treatment programs supported through 
PEPFAR. This is the second of three reports responding to your 
request. We recently issued a related report on PEPFAR treatment 
program costs[Footnote 6] and will soon issue a report on PEPFAR 
supply chains. In this report, we examine (1) PEPFAR treatment program 
results and how OGAC measures them and (2) PEPFAR assistance to 
improve partner countries' monitoring and evaluation (M&E) systems. We 
also provide information--related to treatment program retention, 
patient-level health outcomes, and strengths and weaknesses of partner-
country M&E systems--drawn from studies of treatment programs in 
PEPFAR partner countries.[Footnote 7] 

To address our objectives, we reviewed PEPFAR guidance issued by OGAC 
regarding PEPFAR indicators, definitions, and planning and reporting 
requirements, and we reviewed prior GAO reports on PEPFAR. We examined 
PEPFAR partnership frameworks and PEPFAR's most recent operational 
plans and performance reports as well as data provided by OGAC. We 
determined that these data were sufficiently reliable for the purposes 
of our reporting. We interviewed OGAC, CDC, and USAID officials in 
Atlanta, Georgia, and Washington, D.C., and conducted field work in 
South Africa, Uganda, and Kenya. (See appendix I for further details 
of our scope and methodology.) 

We conducted this performance audit from May 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

OGAC is responsible for establishing overall PEPFAR policy and program 
strategies and allocating funds from the Global Health and Child 
Survival account to PEPFAR implementing agencies, primarily CDC and 
USAID.[Footnote 8] These agencies execute PEPFAR program activities 
through agency headquarters offices and in-country interagency teams 
(PEPFAR country teams) and their implementing partners[Footnote 9] in 
the 33 countries and three regions with PEPFAR-funded programs as of 
fiscal year 2012.[Footnote 10] OGAC coordinates these activities 
through its approval of operational plans, which document work plans, 
budgets, and the anticipated results of HIV/AIDS-related programs. 
OGAC also provides annual guidance to PEPFAR country teams on how to 
develop and submit operational plans. For fiscal years 2009 through 
2012, OGAC approved country operational plan budgets totaling over $16 
billion. Country operational plan activities fit broadly in three 
areas: prevention, treatment, and care. Other program budget areas are 
laboratory infrastructure, strategic information, and health systems 
strengthening. 

To promote a more sustainable approach to combating HIV/AIDS, 
characterized by PEPFAR countries' strengthened capacity, ownership, 
and leadership, the 2008 Leadership Act authorized the U.S. government 
to establish partnership frameworks with partner countries.[Footnote 
11] These frameworks are 5-year joint strategic agreements for 
cooperation between the U.S. government and partner governments to 
combat HIV/AIDS in the partner country through technical assistance 
and support for service delivery, policy reform, and coordinated 
funding commitments.[Footnote 12] As of February 2013, the U.S. 
government had signed 22 PEPFAR partnership frameworks.[Footnote 13] 
According to OGAC guidance, a key expectation of the frameworks is 
that partner-country governments will become better prepared to assume 
primary responsibility for their responses to HIV/AIDS. Moreover, 
PEPFAR's 2012 "blueprint"[Footnote 14] defines country ownership as 
the end state in which partner countries lead, manage, and coordinate 
the efforts needed to ensure that the AIDS response is effective, 
efficient, and durable. 

PEPFAR supports a broad continuum of HIV care and treatment services 
in partner countries. This continuum begins with HIV testing and the 
counseling given to patients learning their HIV status. If patients 
are HIV positive, their eligibility for treatment must be determined 
on the basis of clinical criteria (symptoms associated with HIV), 
laboratory criteria (strength of patients' immune systems),[Footnote 
15] or both clinical and laboratory criteria. The World Health 
Organization (WHO) establishes international guidelines on when to 
initiate treatment for specific groups of HIV-positive people, such as 
adult patients who have never been on treatment, pediatric patients, 
and pregnant and breastfeeding women. In November 2010, WHO updated 
its guidelines by reducing the minimum eligibility threshold in its 
laboratory criteria[Footnote 16] and by recommending treatment for all 
people coinfected with HIV and tuberculosis, thereby expanding the 
number of people eligible for treatment. Based on WHO guidelines, each 
country is expected to establish country-specific guidelines on when 
to initiate treatment for these groups. UNAIDS estimated at the end of 
2011 that, on the basis of WHO's 2010 guidelines, 15 million people in 
low-and middle-income countries needed treatment; of these, an 
estimated 8 million people are on treatment. People who are HIV 
positive but not yet eligible for treatment generally may seek access 
to care and support services as well as regular checkups and 
laboratory monitoring. People eligible for treatment should receive 
antiretroviral (ARV) drugs as well as checkups and monitoring to 
assess the effectiveness of treatment.[Footnote 17] People on 
treatment also receive various care and support services such as 
treatment of opportunistic infections including tuberculosis 
coinfection, nutritional support, and programs to promote adherence to 
treatment and remaining on treatment (patient retention). People on 
treatment are expected to take ARV drugs on a continuing, lifelong 
basis. 

PEPFAR country teams report to OGAC semiannually, usually in May, and 
annually, usually in November, on PEPFAR program results. These 
reports, containing data and narratives, are intended to support 
program monitoring, midcourse correction, and planning for subsequent 
fiscal years. Data on PEPFAR program results also supply information 
for OGAC's annual report to Congress on PEPFAR performance.[Footnote 
18] Since fiscal year 2010, OGAC's Next Generation Indicators 
Reference Guide has provided an updated list of indicators for 
establishing targets and reporting on PEPFAR results.[Footnote 19] 
According to the guidance, these indicators are intended to 
demonstrate progress in the fight against HIV/AIDS while also 
promoting responsible program management. In addition, among other 
things, the guidance establishes a distinction between national 
results and PEPFAR direct results. The guidance defines national 
results as achievements of all contributors to a partner country's 
HIV/AIDS program and defines PEPFAR direct results as achievements of 
the PEPFAR program through its funded activities. (See app. II for a 
summary of OGAC criteria for assessing PEPFAR direct support.) With 
regard to treatment programs, the guidance instructs PEPFAR country 
teams providing direct support for treatment services[Footnote 20] to 
report to OGAC using the PEPFAR direct indicators.[Footnote 21] In 
addition, the guidance directed these country teams, as well as PEPFAR 
country teams providing technical assistance and other support to 
build partner-country capacity for managing treatment programs, to 
report on one national indicator. Table 1 summarizes these indicators. 

Table 1: Key PEPFAR Treatment Indicators: 

Indicator: Number of people currently on treatment[A]; 
Source of data: PEPFAR direct treatment programs; 
Definition: Number of adults and children with advanced HIV infection 
who are currently receiving treatment directly supported by PEPFAR; 
Target: Fiscal year 2012: 5 million; Fiscal year 2013: 6 million. 

Indicator: Treatment coverage; 
Source of data: Partner country treatment programs; 
Definition: Percentage of adults and children with advanced HIV 
infection receiving treatment; 
Target: 80 percent[B]. 

Indicator: Treatment retention; 
Source of data: PEPFAR direct treatment programs; 
Definition: Percentage of adults and children known to be alive and on 
treatment 12 months after starting treatment; 
Target: None[C]. 

Source: GAO synthesis of guidance and information provided by OGAC. 

[A] Since fiscal year 2010, PEPFAR country teams providing direct 
support for treatment services have also been required to report the 
number of adults and children newly enrolled in treatment programs in 
the previous year, disaggregating by age and sex. This indicator 
permits OGAC and country teams to monitor trends in treatment 
initiation and, thus, treatment program expansion. Because data are 
also disaggregated by pregnancy status, the indicator also provides 
data linking treatment programs with programs to prevent the 
transmission of HIV from mother to child. 

[B] Each country sets its own targets for national treatment coverage 
rates. According to UNAIDS, most countries have set 80 percent as 
their target for treatment coverage. 

[C] High retention rates indicate that more people on treatment are 
surviving. However, targets for treatment retention generally are not 
established. 

[End of table] 

Data Indicate Progress in Achieving Treatment Program Results, but 
Some Indicators Have Limitations: 

From fiscal year 2010 through 2012, OGAC reported PEPFAR results in 
terms of three primary indicators: (1) the number of people currently 
on treatment directly supported by PEPFAR (PEPFAR direct number of 
people on treatment), (2) the percentages of eligible people receiving 
treatment in partner countries (national treatment coverage rates), 
and (3) the percentage of adults and children known to be alive and on 
treatment 12 months after starting treatment (PEPFAR direct treatment 
retention rates). However, two of these indicators have limitations 
that could affect their usefulness. Regarding the first indicator, 
although the number of people on treatment directly supported by 
PEPFAR has increased significantly, this indicator alone does not 
provide complete information needed for assessing PEPFAR's 
contributions to partner countries' treatment programs. Regarding the 
third indicator, 10 PEPFAR country teams reported percentages of 
adults and children known to be alive and on treatment 12 months after 
starting treatment that exceeded 80 percent. However, the treatment 
retention data are not always complete and have other limitations, 
which OGAC acknowledged and is taking steps to address. In addition to 
these limitations, OGAC lacks a common set of indicators for 
monitoring quality assurance efforts. Although OGAC indicated in 2010 
that it would establish a common set of indicators to monitor the 
results of PEPFAR's efforts to improve the quality of treatment 
programs, it has not yet done so. 

PEPFAR Exceeded Its Fiscal Year 2012 Direct Treatment Target, but This 
Indicator Does Not Fully Reflect PEPFAR Contributions to Partner-
Country Treatment Programs: 

Responding to treatment-related requirements in the 2008 Leadership 
Act,[Footnote 22] OGAC reports on the number of people currently on 
treatment directly supported by PEPFAR as a key indicator of program 
results.[Footnote 23] This number is calculated by determining the 
number of people who ever started treatment at facilities where PEPFAR 
directly supports treatment services, minus patients who died, stopped 
treatment, transferred out, or have unknown treatment 
outcomes.[Footnote 24] (See app. II for a summary of OGAC guidance on 
determining whether people can be counted as receiving direct services 
through PEPFAR.) PEPFAR met or exceeded annual targets for this 
indicator in fiscal years 2004 through 2012. Currently, this indicator 
is used to track treatment program expansion and to assess progress 
toward PEPFAR's target of providing direct support for treatment for 6 
million people by the end of fiscal year 2013.[Footnote 25] For fiscal 
year 2012, PEPFAR's target for this indicator was 5 million people. 
According to data provided by OGAC, the number of people currently on 
treatment directly supported by PEPFAR has steadily increased from 
about 67,000 people in 11 countries in fiscal year 2004 to more than 
5.1 million in 23 countries in fiscal year 2012.[Footnote 26] (See 
table 2.) 

Table 2: Numbers of People on Treatment Directly Supported by PEPFAR 
and Numbers of PEPFAR Countries Reporting on This Indicator, Fiscal 
Years 2004 through 2012: 

PEPFAR direct number of people currently on treatment; 
2004: 66,700; 
2005: 249,200; 
2006: 541,300; 
2007: 1,091,600; 
2008: 1,743,600; 
2009: 2,485,300; 
2010: 3,195,000; 
2011: 3,905,000; 
2012: 5,057,500. 

Number of PEPFAR country teams reporting on this indicator[A]; 
2004: 11; 
2005: 12; 
2006: 18; 
2007: 24; 
2008: 24; 
2009: 24; 
2010: 23; 
2011: 23; 
2012: 23. 

Source: OGAC. 

Note: Numbers of people on treatment directly supported by PEPFAR have 
been rounded to the nearest hundred. 

[A] In 2004, OGAC reported direct treatment results for 11 PEPFAR 
partner countries and added countries as PEPFAR's investments in them 
increased. Twenty-three PEPFAR country teams provided direct treatment 
services in fiscal year 2012: Botswana, Cambodia, China, Côte 
d'Ivoire, the Democratic Republic of the Congo, the Dominican 
Republic, Ethiopia, Guyana, Haiti, India, Kenya, Lesotho, Mozambique, 
Namibia, Nigeria, Rwanda, South Africa, Swaziland, Tanzania, Uganda, 
Vietnam, Zambia, and Zimbabwe. 

[End of table] 

Furthermore, the number of people on treatment directly supported by 
PEPFAR was about half of the total number of people on treatment in 
all low-and middle-income countries, which UNAIDS estimated at 8 
million in 2011 (see figure 1). 

Figure 1: Number of People on Treatment Directly Supported by PEPFAR, 
Fiscal Years 2004-2012, and Total Number of People on Treatment in 
Low- and Middle-Income Countries, Calendar Years 2004-2011: 

[Refer to PDF for image: stacked line graph] 

Year: 2004; 
Number of people on treatment directly supported by PEPFAR: 66,700; 
Total number of people on treatment in low- and middle-income 
countries: 695,960. 

Year: 2005; 
Number of people on treatment directly supported by PEPFAR: 249,200; 
Total number of people on treatment in low- and middle-income 
countries: 1,311,930. 

Year: 2006; 
Number of people on treatment directly supported by PEPFAR: 541,300; 
Total number of people on treatment in low- and middle-income 
countries: 2,034,000. 

Year: 2007; 
Number of people on treatment directly supported by PEPFAR: 1,091,600; 
Total number of people on treatment in low- and middle-income 
countries: 2,969,040. 

Year: 2008; 
Number of people on treatment directly supported by PEPFAR: 1,743,600; 
Total number of people on treatment in low- and middle-income 
countries: 3,400,640. 

Year: 2009; 
Number of people on treatment directly supported by PEPFAR: 2,485,300; 
Total number of people on treatment in low- and middle-income 
countries: 5,239,280. 

Year: 2010; 
Number of people on treatment directly supported by PEPFAR: 3,195,000; 
Total number of people on treatment in low- and middle-income 
countries: 6,577,030. 

Year: 2011; 
Number of people on treatment directly supported by PEPFAR: 3,905,500; 
Total number of people on treatment in low- and middle-income 
countries: 8,006,540. 

Year: 2012 (projected); 
Number of people on treatment directly supported by PEPFAR: 5,057,500; 
Total number of people on treatment in low- and middle-income 
countries: 9,457,500. 

Source: PEPFAR and UNAIDS data. 

Notes: We estimated the total number of people that received treatment 
in 2012, because UNAIDS data for low-and middle-income countries are 
reported for calendar years and were not available for 2012 at the 
time of this report's publication. PEPFAR data are reported for U.S. 
government fiscal years. 

[End of figure] 

As PEPFAR has begun to shift resources toward providing technical 
assistance and other support to help partner countries build their 
capacity to manage treatment programs, PEPFAR's direct treatment 
indicator has increasingly fallen short of reflecting results of 
PEPFAR's contributions to partner countries' treatment programs. 
Specifically, this indicator does not reflect the expansion of partner 
countries' treatment programs that PEPFAR technical assistance and 
other support for building treatment program management capacity have 
made possible. These efforts include, among others, activities such as 
implementing revised treatment guidelines, assisting partner-country 
district and national health officials with treatment facility 
oversight, and training and mentoring treatment facility staff. In 
several PEPFAR partner countries, for example, PEPFAR implementing 
partners providing direct treatment services have begun transferring 
stable patients to other treatment providers, including an often 
expanding number of local public and private health clinics, many of 
which receive PEPFAR-funded technical assistance and other support. In 
part because of this PEPFAR assistance, these providers also have 
begun increasing the number of people they enroll in treatment. In 
such cases, the PEPFAR direct treatment indicator may not account for 
these people. In the past, PEPFAR's direct results often were 
equivalent to the national number of people receiving services, 
including treatment, according to PEPFAR's 2010 Next Generation 
Indicators Reference Guide. However, as PEPFAR increases its efforts 
to build partner-country capacity to manage treatment programs through 
technical assistance and other support, PEPFAR's direct treatment 
indicator alone does not provide complete information for assessing 
PEPFAR's contributions to partner countries' treatment programs. 

PEPFAR's 2010 Next Generation Indicators Reference Guide noted that 
OGAC was working on a method for deriving PEPFAR direct results from 
partner-country national-level indicators but had not yet devised one. 
[Footnote 27] The guidance stated that the new method would take into 
account the percentage of PEPFAR funding that contributes to partner-
country programs. In addition, OGAC and PEPFAR country teams have 
considered other factors to determine PEPFAR's contribution to partner-
country treatment programs. For example, in their fiscal year 2011 
annual reports to OGAC, seven PEPFAR country teams reported the 
proportion of all treatment facilities receiving PEPFAR support or the 
percentage of all patients on treatment directly supported by PEPFAR. 
Some country teams noted that neither method fully accounted for 
PEPFAR's contributions in these countries. As of February 2013, 
according to a senior OGAC official, OGAC had drafted a method for 
representing PEPFAR contributions based on proportional financial 
support to partner-country program results but had not finalized the 
method or revised its guidance to PEPFAR country teams. 

PEPFAR Has Helped Increase Partner Countries' National Treatment 
Coverage Rates: 

Increases in the number of people on treatment have helped improve 
partner countries' national treatment coverage rates--generally 
defined as the percentage of eligible people receiving treatment. 
[Footnote 28] According to the most current UNAIDS and PEPFAR data, 
[Footnote 29] 8 of the 23 countries where PEPFAR directly supported 
treatment services in 2011 achieved estimated treatment coverage rates 
of 80 percent or more (see table 3).[Footnote 30] Although the 
remaining 15 countries fell short of this target, almost all of these 
countries have increased their estimated treatment coverage rates 
since 2009, according to our analysis of UNAIDS data. 

Table 3: Estimated PEPFAR Partner-Country Treatment Coverage Rates, 2011: 

Partner countries where PEPFAR supports direct treatment services: 

20-39%: 
China[A]; 
Democratic Republic of the Congo[B]; 
Nigeria. 

40-59%: 
Côte d'Ivoire; 
Ethiopia; 
Haiti; 
Lesotho; 
Mozambique; 
Tanzania; 
Uganda; 
Vietnam. 

60-79%: 
Kenya; 
Malawi; 
South Africa; 
Zimbabwe. 

80% or more: Partner countries where PEPFAR supports direct treatment 
services: 
Botswana; 
Cambodia; 
Dominican Republic; 
Guyana; 
Namibia; 
Rwanda; 
Swaziland; 
Zambia. 

Partner countries where PEPFAR provides only technical assistance and 
other support for treatment: : 

Less than 20%: 
South Sudan. 

20-39%: 
Angola; 
Indonesia; 
Kazakhstan[C]; 
Kyrgyzstan[C]; 
Russia; 
Tajikistan[C]; 
Ukraine. 

40-59%: 
Belize[D]; 
Burundi; 
Congo; 
Ghana; 
Guatemala[D]; 
Honduras[D]; 
India; 
Panama[D]; 
Suriname[E]. 

60-79%: 
Costa Rica[D]; 
El Salvador[D]; 
Jamaica[D]; 
Nicaragua[D]; 
Thailand. 

Source: UNAIDS 2012 Global Report: UNAIDS Report on the Global AIDS 
Epidemic and UNAIDS data. 

Note: All PEPFAR countries providing direct treatment services, as 
well as those building partner-country treatment program management 
capacity through technical assistance and other support, report 
estimated national coverage rate information to OGAC. 

[A] Data on China's 2011 national coverage rate were not available 
from UNAIDS. In its fiscal year 2011 report to OGAC, the PEPFAR 
country team provided information on China's 2010 national treatment 
coverage rate. Based on this information, we placed China in the 20-39 
percent category in this table. 

[B] Data on the Democratic Republic of the Congo's 2011 national 
coverage rate were not available from UNAIDS. In its fiscal year 2011 
report to OGAC, the PEPFAR country team provided information on the 
national treatment coverage rate as of June, 2011. Based on this 
information, we placed the Democratic Republic of the Congo in the 20-
39 percent category in this table. 

[C] PEPFAR's Central Asian region comprises Kazakhstan, Kyrgyzstan, 
Tajikistan, Turkmenistan, and Uzbekistan. Treatment coverage data were 
not available for Turkmenistan and Uzbekistan. 

[D] PEPFAR's Central American region comprises Belize, Costa Rica, El 
Salvador, Guatemala, Honduras, Nicaragua, and Panama. PEPFAR funding 
for Belize is approved jointly through the Caribbean and Central 
American regional operational plans. 

[E] PEPFAR's Caribbean region comprises Antigua and Barbuda, the 
Bahamas, Barbados, Belize, Dominica, Grenada, Jamaica, Saint Kitts and 
Nevis, Saint Lucia, Saint Vincent, Suriname, and Trinidad and Tobago. 
Of these countries, treatment coverage data were available for Jamaica 
and Suriname only. 

[End of table] 

PEPFAR Data on Treatment Retention Indicate Some Progress but Have 
Limitations: 

Since fiscal year 2010, OGAC has required PEPFAR country teams 
providing direct support for treatment services to track treatment 
retention rates, an indicator defined as the percentage of adults and 
children known to be alive and on treatment 12 months after starting 
treatment. In addition to being an essential indicator of treatment 
program outcomes--a higher retention rate indicates that more people 
on treatment are surviving--facilities' retention rates are used by 
OGAC and PEPFAR country teams as a proxy indicator of treatment 
program quality. Of the 23 PEPFAR country teams directly providing 
treatment services, 20 provided data on this indicator in their fiscal 
year 2012 reports to OGAC. Ten of the 20 teams reported retention 
rates at or above 80 percent for facilities where PEPFAR implementing 
partners support direct treatment services. (See table 4.) 

Table 4: Reported PEPFAR Treatment Retention Rates, Fiscal Year 2012: 

Less than 60%: 
Côte d'Ivoire. 

60-69%: 
Malawi; 
Tanzania; 
Zambia; 
South Africa. 

70-79%: 
Cambodia; 
Lesotho; 
Mozambique; 
Nigeria; 
Uganda. 

80-89%: 
Democratic Republic of the Congo; 
Ethiopia; 
Guyana; 
Haiti; 
Namibia; 
Swaziland; 
Vietnam. 

More than 90%: 
Botswana; 
China; 
Rwanda. 

Source: GAO analysis of PEPFAR country teams' fiscal year 2011 and 
2012 annual reports and information provided by OGAC. 

Note: PEPFAR country teams for the Dominican Republic, Kenya, and 
Zimbabwe reported that treatment retention data were not available for 
fiscal year 2012. 

[End of table] 

However, PEPFAR patient retention data have several key limitations. 

* The data are not always complete. PEPFAR's reported retention rates 
reflect only the rates at facilities where PEPFAR directly supported 
treatment services and that were able to properly collect and report 
retention data. In addition, in their fiscal year 2012 reports to 
OGAC, three PEPFAR country teams reported that data for this indicator 
were not available. Several country teams noted problems in obtaining 
data from partner-country systems or from all sites where PEPFAR 
directly supports treatment services. Several country teams also 
reported concerns about data quality, including limited understanding 
of how to collect these data. 

* Methods and definitions vary. For example, PEPFAR country teams 
accounted for patients transferring to or from treatment facilities 
differently. In addition, country teams used different definitions to 
count numbers of patients lost to follow-up (i.e., those with unknown 
outcomes, including possible death, treatment cessation, or self-
transfer to another treatment facility). Under the current WHO 
definition, a patient may be considered lost to follow-up 90 days 
after the last scheduled appointment, but the definition may be 
adjusted depending on the stage of a patient's treatment. 

* Data on treatment retention are rarely available for key 
populations, including children and adolescents, injecting drug users, 
men who have sex with men, and sex workers. These populations are at 
higher risk for HIV infection and may face specific challenges that 
make it more difficult to retain them in treatment programs. 

* Few data on long-term retention (after 24 months from the start of 
treatment) are available. Although OGAC guidance encourages PEPFAR 
country teams to use data for cohorts of patients to track retention 
and survival at 24, 36, and 48 months, PEPFAR does not have a 
retention indicator that extends beyond 12 months.[Footnote 31] 

OGAC officials stated that OGAC has taken several steps to improve the 
fiscal year 2012 PEPFAR treatment retention data. First, OGAC 
clarified guidance to PEPFAR country teams regarding how to calculate 
and report on this indicator. Second, PEPFAR implementing agencies 
conducted data quality assessments in three PEPFAR countries. As a 
result, according to OGAC officials, three more PEPFAR country teams 
were able to report on the treatment retention indicator in fiscal 
year 2012 than in fiscal year 2011. Furthermore, OGAC officials stated 
that data completeness is a priority for the current fiscal year 2013 
and that they will help PEPFAR country teams with reporting retention 
data. 

In addition to routinely reported information on treatment program 
results, various studies of treatment programs, although they may not 
represent national treatment program conditions and may be limited by 
incomplete data, provide information that can be useful for improving 
the results of treatment programs. Appendix III contains examples of 
information provided by studies we identified. 

PEPFAR Supports Efforts to Improve Treatment Program Quality, but OGAC 
Has Not Established Indicators to Assess Results: 

PEPFAR country teams engage in a number of activities, often 
characterized as technical assistance or support, whose aim is to 
assure the quality of treatment programs. Seventeen of the 22 
countries with PEPFAR partnership frameworks identified efforts to 
improve treatment program quality as a key goal shared by partner 
countries and PEPFAR. However, OGAC has not established a common set 
of indicators to assess results of these activities. We identified 
several examples of quality assurance activities, such as: 

* developing and implementing partner-country quality improvement 
strategies, including roles and responsibilities for health facility 
supervision; 

* establishing treatment site-level performance improvement plans, 
including quality improvement council meetings to identify solutions 
to problems affecting service quality; and: 

* training health facility managers and staff to track and use 
facility-level performance data. 

OGAC's 2010 Next Generation Indicators Reference Guide sought to 
emphasize program quality indicators to help strengthen partner 
countries' HIV/AIDS programs. To this end, the guidance added patient 
retention rate to the list of essential, reported treatment program 
indicators. The same guidance also recommended tracking data for 
several indicators of treatment program quality--for example, the 
number of patients with a documented CD4 or viral load test, the 
number of patients who have attended the recommended number of 
clinical visits, and the percentage of health facilities providing 
treatment using CD4 monitoring in line with partner-country guidelines 
or policies. 

In the three countries we visited, we found that PEPFAR implementing 
partners were using a wide range of indicators to report on their 
quality assurance activities. Examples of such indicators include 
percentages of (1) HIV-positive patients assessed for treatment 
eligibility, (2) patients on treatment who adhere to the dosing 
instructions and other requirements for taking ARV medicines, and (3) 
patients with good clinical outcomes. However, even where indicators 
in the three countries were generally the same, definitions varied 
slightly. For example, in reporting on the indicator of appointments 
kept, three PEPFAR implementing partners--all providing quality 
assurance assistance to treatment facilities in the same country--used 
two different definitions. One implementing partner reported on the 
percentage of HIV-positive patients who kept their appointments in the 
previous month or quarter, while the other two implementing partners 
reported on the percentage of HIV-positive patients who missed their 
appointments. 

OGAC's 2010 Next Generation Indicators Reference Guide stated that 
additional guidance on quality assurance indicators for PEPFAR 
implementing agencies would be forthcoming. However, as of February 
2013, OGAC has not issued this additional guidance. The lack of PEPFAR-
wide guidance on quality assurance indicators and definitions inhibits 
development of standardized measurement tools used by PEPFAR country 
teams to monitor treatment facilities supported by PEPFAR and 
ultimately track the results of PEPFAR's quality assurance efforts, 
including technical assistance and other support. 

PEPFAR Supports Countries' M&E Systems, but Weaknesses Limit the 
Timeliness and Completeness of Treatment Results Data: 

To track the results of partner-country treatment programs and to help 
ensure that they are effective, PEPFAR supports countries' monitoring 
and evaluation (M&E) systems. While some progress has been made in 
expanding and upgrading them, these systems often are unable to 
produce timely and complete treatment data, limiting their usefulness 
for managing programs and reporting program results. PEPFAR country 
teams fulfill many M&E functions at facilities where PEPFAR supports 
direct treatment services, and they assist partner countries in 
carrying out their M&E responsibilities by providing staff, training, 
and technical assistance and other support. Nevertheless, partner 
countries' M&E systems continue to face a number of weaknesses. 
Consequently, PEPFAR country teams primarily use data drawn from 
PEPFAR-specific systems to report on PEPFAR treatment program results. 
OGAC has not yet issued guidance needed to support PEPFAR's continued 
progress in transitioning to using partner-country M&E systems for 
program management and results reporting. 

PEPFAR Provides Technical Assistance, Training, and Staff to 
Strengthen Partner Countries' M&E Systems: 

Among other activities, OGAC technical guidance to PEPFAR country 
teams calls for the development of partner countries' M&E systems, 
among other activities. Fully functioning M&E systems are essential 
for effective patient monitoring and patient management[Footnote 32] 
and also generate data that PEPFAR country teams and partner countries 
need to track treatment program results.[Footnote 33] All countries 
and regions with PEPFAR partnership frameworks identified 
strengthening M&E systems as a key goal shared by partner countries 
and PEPFAR.[Footnote 34] To support this goal, according to fiscal 
year 2012 PEPFAR operational plans, PEPFAR country teams provide, 
among other things, technical assistance, training, and staff to 
treatment facilities, district health offices, and national ministries 
of health to collect, aggregate, and report treatment program 
information through partner countries' M&E systems (see sidebar). This 
support included recruiting, mentoring, and training health facility 
staff and district health officials responsible for collecting, 
analyzing, aggregating, and reporting data through the country's M&E 
system. PEPFAR country teams were also assisting partner countries 
with conducting surveys and surveillance, such as those needed to 
estimate the number of people with advanced HIV infection. In 
addition, according to OGAC officials, as of February 2013, 15 PEPFAR 
partner countries have expressed interest in a single, open-source 
data system, and several of these countries have started 
implementation. This health information software tool supports 
collecting, analyzing, and reporting national health data, including 
data on treatment programs. According to these officials, PEPFAR will 
support partner-country expansion of the software tool to include 
PEPFAR and Global Fund reporting in two additional PEPFAR partner 
countries. 

Weaknesses Remain in Partner Countries' M&E Systems: 

Side bar: 2 photographs: 

The photo above shows a patient file and paper register used at a 
primary health care facility receiving PEPFAR technical assistance in 
Mpumalanga province, South Africa. The photo below shows three data 
capturers entering patient data into an electronic database at a 
health facility in Kampala, Uganda. children and women. 

Source: GAO. 

[End of side bar] 

PEPFAR country teams have documented weaknesses in a number of partner 
countries' M&E systems, which are at various stages of implementation. 
Our review of PEPFAR country teams' fiscal year 2011 annual reports to 
OGAC and fiscal year 2012 operational plans identified two key 
challenges for partner countries' M&E systems. 

Partner countries' M&E systems often are unable to produce complete 
and timely data, thus limiting their usefulness for patient, clinic, 
or program management. In their 2011 annual reports to OGAC, 12 PEPFAR 
country teams cited timeliness of partner-country treatment program 
data as a challenge, often because partner-country reporting time 
frames differed from the U.S. government fiscal year. In addition, 
three PEPFAR country teams noted that data provided by partner 
countries' M&E systems were incomplete--not all provinces or treatment 
facilities reported data into the system. In addition, one PEPFAR 
country was not able to collect data on the number of patients 
currently receiving treatment, because the partner country provided 
data only on the cumulative number of patients who had ever started 
treatment; the country team noted that this had the likely effect of 
inflating the partner country's treatment coverage rate. 

Furthermore, partner-country health officials, often lacking technical 
capacity, do not always use available data for decision making. Our 
review of the PEPFAR country teams' operational plans found that 23 
teams cited the need to improve data use at treatment facilities or 
other levels of the health care system. For example, one country team 
reported that partner-country health officials tended to focus on data 
collection for reporting rather than for policy, planning, and program 
decision making. Another country team reported that lack of data 
reporting by treatment facilities limited analysis of treatment 
patients across facilities, and a third country team noted that human 
resource limitations and weak research capacity impeded use of M&E 
data. In addition, studies of PEPFAR partner countries' M&E systems 
that we identified provided additional information; appendix III 
provides a summary of this information. 

PEPFAR Country Teams Aim to Integrate M&E Systems, but OGAC Has Not 
Established Minimum Standards to Guide This Effort: 

Because of the limitations associated with the data from national M&E 
systems, PEPFAR country teams primarily use data drawn from systems 
created specifically for reporting PEPFAR treatment program results. 
OGAC's 2010 Next Generation Indicators Reference Guide states that 
PEPFAR country teams may need to rely on these systems in the short 
term but should continue working to integrate these systems into 
partner countries' M&E systems. Our review of PEPFAR country teams' 
operational plans found that PEPFAR country teams maintained PEPFAR 
program performance management systems to routinely collect, compile, 
and analyze patient monitoring and management data from the health 
facilities where PEPFAR directly supports treatment services. In 
addition, PEPFAR country teams use these data to generate their 
semiannual and annual reports to OGAC. PEPFAR country teams supplement 
information from their own systems with data from partner countries' 
M&E systems, including numbers of patients on treatment and rates of 
treatment coverage. 

OGAC's 2010 Next Generation Indicators Reference Guide recommends 
several indicators for tracking partner-country outcomes related to 
strengthening health systems, such as the existence of M&E plans and 
the percentage of health facilities with record-keeping systems for 
monitoring HIV/AIDS programs.[Footnote 35] In addition, OGAC's 
technical guidance to country teams for developing partner countries' 
M&E systems identifies a number of key efforts, such as developing M&E 
leadership and organizations, improving the policy environment, and 
ensuring the advancement and sustainability of technical capacity in 
PEPFAR partner countries. The technical guidance states that these 
efforts should support national capacity building. However, OGAC has 
not issued guidance identifying minimum standards that data generated 
by partner countries' M&E systems should meet--such as standards 
related to completeness and timeliness--in order for PEPFAR country 
teams to assess, together with partner countries and other donors, 
whether the systems are ready for use in PEPFAR program management and 
results reporting. The lack of such standards leaves uncertain the 
point at which partner-country M&E systems are mature enough for 
PEPFAR to rely on them. This uncertainty is likely to delay 
achievement of PEPFAR's goal of using partner-country M&E systems to 
generate data for PEPFAR treatment program management and reporting. 

Conclusions: 

Ensuring that PEPFAR treatment programs continue to improve and to 
operate as effectively as possible requires careful, complex 
monitoring and evaluation (M&E), not only of program results but also 
of quality assurance efforts. Available data on PEPFAR program results 
show some progress. In particular, PEPFAR and UNAIDS data indicate a 
steady increase in the number of people on treatment, improved 
treatment coverage rates, and high rates of patient retention at many 
facilities. However, PEPFAR's contributions to expansion of partner 
countries' treatment programs are not fully reflected in its results 
data because OGAC's current method for deriving PEPFAR's direct 
treatment indicator does not fully account for PEPFAR's efforts to 
improve partner countries' capacity to manage their treatment 
programs. This limits the usefulness of PEPFAR's direct treatment 
indicator for assessing progress toward expanding partner-country 
treatment programs. Furthermore, OGAC has not yet established a common 
set of indicators to measure the results of PEPFAR technical 
assistance and other support intended to improve the quality of 
treatment programs. Lacking a standard set of quality assurance 
indicators, PEPFAR is limited in its ability to track the results of 
PEPFAR's quality assurance efforts. 

As PEPFAR continues to shift responsibility for managing treatment 
programs to partner countries, those countries will need robust M&E 
systems to generate the data that are indispensable for ensuring 
effective treatment and efficient program management. PEPFAR has 
dedicated resources specifically for these efforts, but problems with 
untimely and incomplete data collection, as well as with data use, 
persist. As a result, PEPFAR has to rely on the M&E systems its 
implementing partners have developed rather than on country-managed 
systems for collecting and reporting results data. In its guidance, 
OGAC has not yet established minimum standards that data generated by 
partner countries' M&E systems should meet in order for PEPFAR country 
teams to assess these systems. Without such standards, uncertainty 
remains as to when partner-country M&E systems will be ready to be 
integrated with PEPFAR systems, thus delaying the achievement of 
PEPFAR's goal of using partner-country M&E system data for PEPFAR 
treatment program management and reporting. 

Recommendations for Executive Action: 

To ensure the outcomes and quality of treatment programs supported by 
PEPFAR, we recommend that the Secretary of State direct the U.S. 
Global AIDS Coordinator to take the following three actions in 
collaboration with PEPFAR implementing agencies: 

* develop a method that better accounts for PEPFAR's contributions to 
partner-country treatment programs; 

* establish a common set of indicators to measure the results of 
treatment program quality improvement efforts; and: 

* establish a set of minimum standards for data generated by partner 
countries' M&E systems, to enable PEPFAR country teams to assess those 
systems' readiness for use in treatment program management and 
reporting. 

Agency Comments and Our Evaluation: 

Weprovided a draft of this reportto State, USAID, and CDC. Responding 
jointly with CDC and USAID, Stateprovided written comments(see appendix 
IV for a copy of these comments). State and CDC also provided technical 
comments and supplementary information relating to our findings and 
recommendations. In response to the technical comments, we incorporated 
changes to the draft report, as appropriate. After reviewing the 
supplementary information, we clarified our findings and recommendations 
relating to PEPFAR’s direct treatment indicator and its support for 
partner countries’ M&E systems."

In its written comments, State generally agreed with our three 
recommendations. First, State affirmed that it supports our 
recommendation to develop a method for fully accounting for PEPFAR's 
contributions to partner-country treatment programs. Observing that 
PEPFAR's direct treatment indicator was intended to capture only 
essential components of direct treatment services, State noted that 
PEPFAR has recently begun an effort to revise its monitoring, 
evaluation, and reporting framework, including an expansion of 
indicators that would allow for implementing partners to report on 
their efforts to help partner countries build capacity and develop 
sustainable treatment programs. Second, State also agreed with the 
finding leading to our recommendation regarding the development of 
indicators to measure the results of treatment program quality 
improvement efforts. State cited the need for a harmonized PEPFAR 
strategy on treatment quality, including key indicators, and noted 
steps it is taking to develop such a strategy. In addition, stressing 
that treatment retention indicators are relatively new and difficult 
to operationalize, State detailed steps PEPFAR is taking to help 
improve treatment retention measurement, evaluation, and performance. 
Third, State noted that PEPFAR supports the strengthening of partner 
country reporting systems and works with partner countries to help them 
develop such systems, both to support national programs as well as to 
provide data for PEPFAR and other donors. As part of these efforts, 
PEPFAR also works with WHO and the Global Fund on system standardization 
and standards for data exchange. State specifically identified an 
indicator developed by WHO for documenting data completeness and 
timeliness and stated that this indicator can be used to monitor efforts 
to develop these reporting systems. We agree that such an indicator 
could be useful for PEPFAR country teams trying to determine when 
partner-country M&E systems are ready to be integrated with PEPFAR systems. 
However, we note that PEPFAR guidance does not instruct country teams to 
use the WHO indicator or any other indicator for this purpose. We 
believe that specifying one or more indicators for PEPFAR country teams 
to use is important to ensure a consistent approach to systems integration 
across the program. Doing this would emphasize for country partners the 
importance of harmonizing M&E systems for mutually beneficial purposes. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to the 
Secretary of State and the U.S. Global AIDS Coordinator. The report 
also will be available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-3149 or gootnickd@gao.gov or contact Marcia 
Crosse at (202) 512-7114 or crossem@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions 
to this report are listed in appendix V. 

Signed by: 

David Gootnick: 
Director, International Affairs and Trade: 

Signed by: 

Marcia Crosse: 
Director, Health Care: 

List of Requesters: 

The Honorable Lamar Alexander: 
Ranking Member: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

The Honorable Tom Coburn, MD: 
Ranking Member: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Richard Burr: 
Ranking Member: 
Subcommittee on Primary Health and Aging: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

The Honorable Michael B. Enzi: 
Ranking Member: 
Subcommittee on Children and Families: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

The Honorable Johnny Isakson: 
Ranking Member: 
Subcommittee on Employment and Workplace Safety: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

In this report, we examine the President's Emergency Plan for AIDS 
Relief's (PEPFAR) (1) treatment program results and how the Department 
of State's (State) Office of the U.S. Global AIDS Coordinator (OGAC) 
measures them and (2) assistance to improve partner countries' 
monitoring and evaluation (M&E) systems. To address both of these 
objectives, we collected and analyzed information from the following 
sources: interviews and fieldwork; guidance documents and past GAO 
work; PEPFAR partnership frameworks, operational plans, and 
performance reports; and studies of treatment programs. 

We interviewed officials from OGAC, the Department of Health and Human 
Services' (HHS) Centers for Disease Control and Prevention (CDC), and 
the U.S. Agency for International Development (USAID) in Washington, 
D.C., and Atlanta, Georgia. We also conducted fieldwork in three 
PEPFAR partner countries--Kenya, South Africa, and Uganda--in June 
2012 to obtain information on PEPFAR efforts to support partner-
country treatment program outcomes and quality as well as challenges 
faced by PEPFAR implementing agencies and stakeholders. We selected 
these countries based on program size, availability of cost data, 
travel logistics, and other factors. We interviewed U.S. agency 
officials, representatives of key implementing partners, and partner 
government health officials and conducted visits to selected treatment 
facilities in these countries. 

We reviewed guidance provided to PEPFAR country teams[Footnote 36] by 
OGAC in collaboration with CDC, USAID, and other PEPFAR implementing 
agencies and previous GAO work to identify criteria related to 
requirements for collecting, validating, and reporting treatment 
program results and their measures.[Footnote 37] These guidance 
documents also provided criteria for examining PEPFAR assistance to 
improve partner countries' M&E systems. Guidance documents included 
OGAC's 2010 Next Generation Indicators Reference Guide, issued in 
August 2009; fiscal years 2011 and 2012 operational plan guidance and 
associated technical considerations; and annual and semiannual 
performance reporting guidance. We also consulted relevant guidance 
and reports issued by international organizations such as the United 
Nations Joint Programme on HIV/AIDS (UNAIDS) and the World Health 
Organization (WHO). These included Global Report: UNAIDS Report on the 
Global AIDS Epidemic, 2012 and Organizing Framework for a Functional 
National HIV Monitoring and Evaluation System as well as WHO's 2010 
Antiretroviral Therapy for HIV Infection in Adults and Adolescents: 
Recommendation for a Public Health Approach and 2006 Patient 
Monitoring Guidelines for HIV Care and Antiretroviral Therapy. We also 
reviewed WHO's 2011 Retention in HIV Programmes: Defining the 
Challenges and Identifying Solutions. 

We examined PEPFAR operational plans, performance reports, and 
partnership frameworks. First, to identify key PEPFAR treatment 
program goals--including those related to treatment program 
supervision, M&E systems, and quality assurance--we reviewed 
partnership framework agreements between the United States and 22 
PEPFAR partner countries and regions.[Footnote 38] Next, to identify 
ongoing and planned PEPFAR activities, as well as challenges, related 
to treatment programs and M&E systems, we reviewed relevant sections 
of PEPFAR operational plans for fiscal year 2012 for the 33 PEPFAR 
countries and three regions. We documented instances where the 
operational plans addressed these activities, and we summarized the 
information for all PEPFAR countries and regions. Finally, to identify 
PEPFAR treatment program results, as well as challenges associated 
with these program activities and PEPFAR assistance to improve partner 
countries' M&E systems, we reviewed PEPFAR semiannual and annual 
reports for fiscal years 2011 and 2012 submitted by PEPFAR country 
teams to OGAC as well as aggregate PEPFAR data provided to us by OGAC. 
In fiscal years 2011 and 2012, the 33 PEPFAR country and three 
regional teams provided semi-annual and annual reports to OGAC 
containing data and narrative descriptions of each country's or 
region's PEPFAR activities. We reviewed all PEPFAR countries' and 
regions' fiscal year 2011 annual reports to identify strengths and 
weaknesses of partner countries' M&E systems. In addition, the 23 
PEPFAR country teams that provided direct support for treatment 
services reported PEPFAR treatment indicator data and narrative 
descriptions to OGAC regarding (1) the number of adults and children 
with advanced HIV infection who are currently receiving treatment 
directly supported by PEPFAR (PEPFAR direct number of people currently 
on treatment); (2) the percentage of adults and children with advanced 
HIV infection who are receiving treatment in partner countries' 
treatment programs (national coverage rates); and (3) the percentage 
of adults and children known to be alive and on treatment 12 months 
after starting treatment directly supported by PEPFAR (PEPFAR direct 
treatment retention rates). We reviewed and summarized the 23 country 
teams' narrative descriptions accompanying each of these three PEPFAR 
indicators for information related to treatment program results and 
assistance to improve PEPFAR partner countries' M&E systems. 

In the case of the number of people on treatment directly supported by 
PEPFAR, to show changes over time, we analyzed aggregate data for 
fiscal years 2004 through 2012 provided by OGAC, which it derived from 
PEPFAR country teams' semi-annual and annual reports. We previously 
have reviewed OGAC guidance and procedures for collecting, analyzing, 
and assessing these data.[Footnote 39] In addition, to identify 
factors considered by PEPFAR country teams when reporting on PEPFAR 
treatment program results, we reviewed the narrative descriptions 
provided in these PEPFAR country teams' fiscal year 2011 annual 
reports. On the basis of these reviews, as well as interviews with 
OGAC officials, we determined that the PEPFAR data on number of people 
on treatment were sufficiently reliable for reporting totals rounded 
to the nearest hundred. In addition, to illustrate PEPFAR's 
contribution to the number of people on treatment in low-and middle-
income countries, we obtained data on the numbers of people on 
treatment in all low-and middle-income countries from the UNAIDS 
website [hyperlink, http://www.unaids.org] for calendar years 2004 
through 2011. UNAIDS sets international standards for these data, 
which it collects from national governments and uses to report on 
estimated numbers of people on treatment; therefore, we deemed them 
sufficiently reliable for the purposes of our reporting. Data on the 
number of people on treatment in low-and middle-income countries were 
not available for 2012 at the time of this report's publication. To be 
able to illustrate PEPFAR's contribution to the number of people on 
treatment in low-and middle-income countries in 2012, we derived a 
rough estimate based on changes in PEPFAR's and UNAIDS's reported 
numbers of people on treatment from 2010 to 2011. We observed that the 
number of people on treatment directly supported by PEPFAR made up 
about half of the increase in the number of people on treatment in 
low- and middle-income countries in these years. For fiscal years 2011 
to 2012, the number of people on treatment directly supported by 
PEPFAR increased by about 1.2 million; if we had assumed that this 
number continued to make up half of the increase in low-and middle-
income countries in these years, the estimated increase would have 
been 2.4 million. Thus, our estimate of an increase of about 1.5 
million people on treatment from 2011 to 2012--leading to an estimate 
of 9.5 million people on treatment in low-and middle-income countries 
in those years--represents a conservative projection of possible 
scenarios. 

In the case of partner countries' treatment coverage rates, we 
analyzed data provided in UNAIDS's 2012 Global Report: UNAIDS Report 
on the Global AIDS Epidemic as well as data available for 2009 through 
2011 on UNAIDS's website [hyperlink, http://www.unaids.org]. UNAIDS 
sets international standards for these data, which it collects from 
national governments and uses to report on national treatment coverage 
rates. On the basis of review of treatment coverage rates reported by 
PEPFAR country teams to OGAC, as well as discussions with OGAC 
officials, we determined that UNAIDS's treatment coverage data were 
the most complete and current data available and were thus 
sufficiently reliable for the purposes of our reporting. 

In addition to information provided by PEPFAR country teams and 
reported by OGAC, we also drew on information from a selected set of 
studies of treatment programs and M&E systems in PEPFAR partner 
countries. We used the information from these studies to identify 
illustrative examples of factors affecting treatment program outcomes 
and M&E system strengths and weaknesses. Appendix III contains 
examples of information provided by studies we reviewed. 

First, although we did not intend to develop an exhaustive list of all 
available studies, we took a number of steps to identify relevant and 
up-to-date studies assessing treatment programs and M&E systems in 
PEPFAR countries. These studies included: (1) studies collected under 
our previous review of evaluations of PEPFAR programs; (2) studies 
provided by PEPFAR country teams in the three countries we visited; 
(3) articles published in special issues of Health Affairs and the 
Journal of Acquired Immune Deficiency Syndromes dedicated to PEPFAR 
programs, both published in 2012; and (4) a citations review for 
PEPFAR public health evaluations, evaluations provided by CDC 
headquarters, and relevant articles appearing in Health Affairs and 
the Journal of Acquired Immune Deficiency Syndromes from 2009 through 
2012. We identified more than 200 studies addressing our objectives. 

To perform our review, we first reviewed the studies' titles and 
abstracts to categorize each study according to one or more topics, 
such as M&E systems or treatment program retention. Focusing on 
studies that fell into categories related to M&E systems and to 
treatment program retention and patient-level outcomes, we then 
reviewed key sections--such as findings and conclusions--of each study 
to identify common themes. We also targeted studies that addressed 
topics covered in the body of the report, such as factors affecting 
treatment program retention and loss to follow-up and strengths and 
weaknesses of M&E systems. Having identified subsets of relevant 
studies, we reviewed them in more depth to verify our initial 
judgments about the studies' findings and to select illustrative 
examples of the themes we had identified. This additional review 
included the development of narrative work papers synthesizing and 
categorizing more detailed findings and results from the selected 
studies. We then presented these findings and examples in appendix 
III. Our analysis is not a summary of the full set of studies we 
identified in the initial phase but rather a presentation of several 
key, high-level results derived from a select set of studies in both 
areas, supported with citations to illustrative studies. 

We conducted this performance audit from May 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Criteria for Assessing PEPFAR Direct Support: 

The Department of State's (State) Office of the U.S. Global AIDS 
Coordinator's (OGAC) Next Generation Indicators Reference Guide, 
effective beginning in fiscal year 2010 and updated in February 2013, 
provides a list of indicators for setting targets, monitoring results, 
and reporting to OGAC. The guidance distinguishes between President's 
Emergency Plan for AIDS Relief (PEPFAR) direct indicators and national 
indicators. PEPFAR direct indicators describe the results of PEPFAR 
programs through its funded activities. National indicators describe 
the achievements of all contributors--including public and private 
sector organizations and other donors--to a partner country's HIV/AIDS 
programs. 

Furthermore, OGAC's guidance provides a checklist for determining 
whether a site-specific service supported by PEPFAR can be counted in 
reporting on PEPFAR direct indicators. According to the guidance, to 
be characterized as PEPFAR direct, an activity must fulfill at least 
one criterion in each of the two panels shown in table 5. If an 
activity meets at least one of each set of criteria, PEPFAR support is 
assumed to be direct and to likely provide sufficient support for 
claiming 100 percent of the site-specific results. If an activity 
meets a criterion in only one of the panels, PEPFAR support may be 
insufficient for claiming 100 percent of site-specific results. In 
that case, country teams must (1) determine whether there is 
sufficient justification to claim the results as direct and, if there 
is, (2) justify the method used to estimate the appropriate fraction 
of the total commensurate with PEPFAR support to the site and (3) 
document the estimation procedures used. 

Table 5: Checklist to Determine Whether to Count Service Delivery 
Support as PEPFAR Direct: 

Assessment criteria: 
Panel 1: Compared with other donors/partners, the dollar value that 
PEPFAR invests at the service delivery site is substantial; 
Panel 2: Quality prevention, care, and/or treatment services at the 
site(s) would not occur in the absence of PEPFAR support. 

Assessment criteria: 
Panel 1: PEPFAR has frequent (i.e., more than 1 day per week) contact 
with service delivery site personnel, patients, and/or clients; 
Panel 2: The quality of the services provided at the service delivery 
site(s) would be unacceptably low without PEPFAR support. 

Assessment criteria: 
Panel 1: PEPFAR staff regularly assists with essential M&E functions 
provided at the service delivery site; 
Panel 2: The support provided represents a substantial contribution 
toward sustainability of services at the service delivery site(s). 

Source: PEPFAR Next Generation Indicators Reference Guide, version 
1.1, August 2009. 

[End of table] 

[End of section] 

Appendix III: Summary of Studies: 

In addition to analyzing information provided by President's Emergency 
Plan for AIDS Relief (PEPFAR) country teams and reported by the 
Department of State's (State) Office of the U.S. Global AIDS 
Coordinator (OGAC), we also performed reviews of a selected set of 
studies.[Footnote 40] Although these studies may not represent 
national treatment program conditions and may be limited by incomplete 
data, they provide additional information related to treatment 
retention and patient-level health outcomes, as well as strengths and 
weaknesses of partner-country monitoring and evaluation (M&E) systems. 
(See appendix I for information on how we identified and used these 
studies for the purposes of our reporting.) 

Treatment Retention and Patient-Level Outcomes: 

Treatment retention rates[Footnote 41] reported by PEPFAR country 
teams indicate some progress but have certain limitations related to 
data completeness and varying methods and definitions. Treatment 
program studies--which use treatment facility data not routinely 
reported by PEPFAR country teams to OGAC--also provide information 
about treatment retention, adherence, and patient loss-to-follow-up. 
[Footnote 42] 

Several studies, although not representative of national program 
conditions, identified factors associated with patient loss-to-follow-
up, such as advanced illness and personal economics. They also noted 
measures that could potentially reduce the number of patients lost to 
follow-up and thus increase retention rates, such as starting 
treatment earlier, shortening the distance to health care facilities, 
expanding personal outreach by community health volunteers, and using 
mobile phone messaging for patient follow-up. In addition, several 
studies that we reviewed identified interventions found to be 
successful for reducing the occurrence of drug resistance and 
treatment failure, including (1) increased outreach to patients to 
increase adherence to medications and (2) effective supervision and 
training of lower-level health facility staff. Other studies 
identified factors that may limit the positive health effects of 
treatment, such as poor access to nutrition and clean water. Such 
findings can serve as the basis for adopting new treatment program 
policies and management approaches. 

Incomplete data may limit studies' ability to fully examine patient-
level health outcomes and the factors that affect them. For example, 
we identified one study that reported some positive adult treatment 
program outcomes, based on a nationally representative sample of 
treatment facilities in Rwanda, but noted incomplete data on patient 
weight and CD4 cell count, among other limitations. Another study on 
the prevalence of drug-resistant HIV strains noted that very few 
published data on drug resistance are available, particularly for the 
HIV strains that tend to be prevalent in low-and middle-income 
countries. Devising mitigation strategies for drug resistance can be 
difficult without these data. Likewise, one study noted a lack of data 
on potential adverse effects of treatment on the growth and 
development of pediatric patients. 

Strengths and Weaknesses of Partner-Country M&E Systems: 

Studies of partner countries' M&E systems note that some progress has 
been made in expanding and upgrading these systems. The studies cite 
increased use of electronic systems for health information management 
and reporting. For example, one study found that electronic systems, 
where in use, enabled health facilities to report on health indicators 
more easily as well as to support patient and facility management. 

Nevertheless, these studies also found that partner countries' M&E 
systems are unable to produce timely and complete treatment data, thus 
limiting their usefulness for patient, clinic, or program management. 
These studies cited several challenges, including the following: 

* Human resource limitations can hinder the full use of M&E system 
data. For example, one study noted that health facility staff lacked 
trained data managers to regularly analyze basic data. 

* Data may not always be analyzed when reported from the facility to 
the district; and data may not always be disseminated properly, or may 
not be used in decision making. For example, one study noted limited 
access to data and lack of capacity as factors negatively affecting 
data use. 

* Health facility staff may prioritize data reporting to districts, 
national ministries, and donors over data use, including use for 
improving the quality of treatment services. For example, one study 
found that data were used for decision making at 38 percent of health 
facilities reviewed and in 44 percent of districts reviewed. 

[End of section] 

Appendix IV: Comments from the Department of State: 

United States Department of State: 
Comptroller: 
1969 Dyess Avenue: 
Charleston, SC 29405: 

April 4, 2013: 

Dr. Loren Yager: 
Managing Director: 
International Affairs and Trade: 
Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Dr. Yager: 

We appreciate the opportunity to review your draft report, 
"President's Emergency Plan for AIDS Relief: Shift to Partner-Country 
Treatment Programs Will Require Better Information on Results" GAO Job 
Code 320913. 

The enclosed Department of State comments are provided for 
incorporation with this letter as an appendix to the final report. 

If you have any questions concerning this response, please contact 
Leigh Ann Monk-Reyes, Program Support Officer, Office of the U.S. 
Global AIDS Coordinator at (202)663-2753. 

Sincerely, 

Signed by: 

James L. Millette: 

cc: 

GAO - David Gootnick; 
S/GAC- Eric Goosby; 
State/OIG - Evelyn Klemstine. 

Department of State Comments on GAO Draft Report: 

President's Emergency Plan For AIDS Relief: 

Shift to Partner-Country Treatment Programs Will Require Better
Information on Results (GAO-13-460, GAO Code 320913): 

Thank you for the opportunity to comment on your draft report 
entitled, "President's Emergency Plan For AIDS Relief Shift to Partner-
Country Treatment Programs Will Require Better Information on Results, 
GA0-13-360, Job Code 320913." 

The GAO report included three recommendations for the Department of 
State's Office of the U.S. Global AIDS Coordinator (S/GAC). 

The Department of State's Office of the U.S. Global AIDS Coordinator 
and the PEPFAR implementing agencies appreciate the work conducted by 
the GAO to produce these findings and the report. The PEPFAR program 
is committed to both improving the quality and the use of data to 
better manage programs, optimize impact, and capture PEPFAR's 
contribution to country HIV/AIDS treatment scaleup and outcomes. 
PEPFAR works in partnership with partner countries, the Global Fund 
for AIDS TB and Malaria, the WHO, UNAIDS and other stakeholders to 
achieve ambitious treatment goals within the framework of a country-
owned health response. This requires that PEPFAR harmonize its efforts 
with international partner organizations and countries to support an 
effective monitoring and evaluation system that minimizes overlapping 
mandates and duplicative or burdensome data collection and reporting, 
respecting often constrained human and other resources to carry these 
out. 

First, GAO recommended that S/GAC develop a method that fully accounts 
for PEPFAR's contributions to partner-country treatment programs. 

PEPFAR is committed to supporting national programs and developing 
methods to better account for its contributions to partner-country 
treatment programs. We are supportive of GAO's recommendation that 
PEPFAR's contribution to treatment outcomes be accurately captured and 
reflective of the breadth of activities conducted by partners that 
result in successful treatment. The GAO report correctly notes that 
the currently used treatment indicator associated with service 
delivery does not fully account for PEPFAR's contributions to partner 
country treatment programs, as it was intended to only capture 
essential components of direct treatment services to individuals. 
PEPFAR has just initiated a process for revising its entire 
monitoring, evaluation, and reporting framework. A component of this 
effort is focused on expanding available indicators for documenting 
program activities, allowing partners to report on their work that 
supports district health service delivery supervision, mentoring, 
policy change, as well as institutional capacity building and 
additional activities that are structured to support sustainable 
treatment program development and implementation in partner countries. 
Currently, our standard indicators do not allow for complete 
documentation of the breadth of PEPFAR's technical support for 
treatment. These new measures will provide a more robust means to 
describe the nature of PEPFAR support, and will be a more realistic 
and useful method to measure the full extent of its contributions to 
national and global HIV treatment outcomes. This new framework and 
indicators, designed to support PEPFAR in the next phase of the 
initiative, will become available in the summer 2013. 

Second, GAO recommended that S/GAC establish a common set of
indicators to measure the results of treatment program quality 
improvement efforts. 

As the report notes, PEPFAR does not have a formal PEPFAR-wide 
strategy on monitoring treatment quality although its implementing 
agencies engage in many activities to assure the quality of treatment 
programs, and in many cases have quality strategies and standards. For 
example, tracking performance through periodic site quality monitoring 
using standardized measurement tools, with remediation plans for areas 
needing improvement, is carried out by many PEPFAR programs providing 
HIV clinical services. Data quality assessments to validate site level 
data are also done on a sampling basis. As the PEPFAR program has 
matured and clinical responsibilities are increasingly transferred 
from international partners to local partners and governments, it is 
clear that a harmonized PEPFAR strategy on treatment quality is needed 
to assure and monitor quality of service delivery. As with treatment 
indicators, PEPFAR is committed to supporting, informing, and aligning 
with partner country strategies for monitoring treatment quality and 
effectiveness of measures to improve it. Other stakeholders, including 
the Global Fund, have shared interests and objectives in working with 
partner countries and PEPFAR towards a uniform country plan for 
quality standards. 

Earlier this year, S/GAC established a task team to develop a PEPFAR-
wide quality strategy for clinical programs, including treatment and 
care. The task team is composed of S/GAC staff representing adult and 
pediatric treatment, care, and strategic information. The team is 
working with the PEPFAR Technical Working Groups and implementing 
agencies to develop a quality strategy for clinical programs, 
including the development and refinement of key indicators, standards 
which encompass quality assurance and improvement processes, and 
standardized protocols to ensure quality data that are fundamental to 
these measures. Expected deliverables in 2013 include a stand-alone 
clinical quality strategy document, updates to the PEPFAR Technical 
Considerations, Country Operational Plan guidance, and the new 
indicators guidance. Building the capacity of partner countries to 
design, implement, and support these efforts is a key objective of 
this agenda. 

In addition, given the critical importance of retention to quality 
treatment and care programs, the task team will also develop stand-
alone guidance on retention, with tool-kits to help improve retention 
measurement, evaluation, and performance across PEPFAR. Measures of 
retention are relatively new and complex to operationalize in the 
context of evolving data management systems, decentralization of 
PEPFAR-supported services across public and private sector providers, 
and as the management and oversight functions for treatment support 
transition to partner countries. The forthcoming guidance and toolkits 
aim to allow PEPFAR and partner countries to achieve the greatest 
impact of investments in care and treatment by tracking retention and 
identifying when program adjustments are needed to optimize this. This 
work is linked closely with support for national M&E systems, as noted 
below. Retention, as well as many other outcome measures, is heavily 
dependent on data systems, and until these systems are comprehensive 
and fully implemented, PEPFAR is supporting development of 
representative sampling strategies to provide quality information on 
retention (and other outcome) tracking. 

Third, GAO recommended that S/GAC establish a set of minimum standards 
for data generated by partner countries' M&E systems. 

PEPFAR supports the development and strengthening of partner country 
reporting systems, but does not independently create standards for 
those systems. Instead, consistent with PEPFAR's emphasis on country 
ownership, PEPFAR works with our national partners to help them 
develop robust systems to support the national agenda as well as to 
provide data for PEPFAR and other donor entities. A related indicator 
for use in countries documents data completeness and timeliness among 
the collection of measures published by the WHO for Health Systems 
Strengthening This or a comparable measure can be used by the national 
system to monitor the success of our collaborative efforts. 

One step PEPFAR is taking is to ensure that M&E systems used by host 
countries and those used within the PEPFAR program incorporate the 
flexibility needed to accommodate each other's reporting timeframe. 
This activity is specifically designed to address current issues in 
the non-alignment of USG and host country reporting periods that 
contributes to the 'timeliness' concerns. 

In addition, while PEPFAR believes national ownership is essential, 
PEPFAR also has been a major catalyst in system standardization among 
our partner countries for the last several years, which should help 
ensure that certain minimum standards for data quality are met. Toward 
this end, we have supported the WHO's efforts to develop consensus for 
and publish international standards for data exchange. At an 
implementation level, PEPFAR has been the primary sponsor of the Open 
Health Information Exchange (OpenHIE) initiative, founded on the 
principles of standards-based open source and interoperability. 
Applications range from client-focused systems found in health clinics 
to population-focused systems providing aggregated data for 
management, performance, and public health objectives. The link 
between these two types of systems is critical to aggregate and move 
data among the various administrative levels for analyses. PEPFAR is 
currently working with the Global Fund to support the implementation 
of the District Health Information Software (DHIS) system, the system 
of choice for many of our partner countries, to manage aggregate data 
for national, PEPFAR, Global Fund, and other donor reporting. We 
anticipate that this capability will expand rapidly across countries 
over the next five years, and greatly increase the capacity of our 
partner countries to support the use and reporting of quality data. 

[End of section] 

Appendix V: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

David Gootnick, (202) 512-3149 or gootnickd@gao.gov: 

Marcia Crosse, (202) 512-7114 or crossem@gao.gov: 

Staff Acknowledgments: 

In addition to the contacts named above, Jim Michels (Assistant 
Director), Todd M. Anderson, David Dayton, Brian Hackney, and Grace 
Lui made key contributions to this report. In addition, the following 
GAO staff provided technical assistance and other support: Sada 
Aksartova, Chad Davenport, David Dornisch, Lorraine Ettaro, Katherine 
Forsyth, Kay Halpern, Erika Navarro, and Jane Whipple. 

[End of section] 

Related GAO Products: 

President's Emergency Plan for AIDS Relief: Per-Patient Costs Have 
Declined Substantially, but Better Cost Data Would Help Efforts to 
Expand Treatment. [hyperlink, http://www.gao.gov/products/GAO-13-345]. 
Washington, D.C.: March 15, 2013. 

President's Emergency Plan for AIDS Relief: Agencies Can Enhance 
Evaluation Quality, Planning, and Dissemination. [hyperlink, 
http://www.gao.gov/products/GAO-12-673]. Washington, D.C.: May 31, 
2012. 

President's Emergency Plan for AIDS Relief: Program Planning and 
Reporting. [hyperlink, http://www.gao.gov/products/GAO-11-785]. 
Washington, D.C.: July 29, 2011. 

Global Health: Trends in U.S. Spending for Global HIV/AIDS and Other 
Health Assistance in Fiscal Years 2001-2008. [hyperlink, 
http://www.gao.gov/products/GAO-11-64]. Washington, D.C.: October 8, 
2010. 

President's Emergency Plan for AIDS Relief: Efforts to Align Programs 
with Partner Countries' HIV/AIDS Strategies and Promote Partner 
Country Ownership. [hyperlink, 
http://www.gao.gov/products/GAO-10-836]. Washington, D.C.: September 
20, 2010. 

President's Emergency Plan for AIDS Relief: Partner Selection and 
Oversight Follow Accepted Practices but Would Benefit from Enhanced 
Planning and Accountability. [hyperlink, 
http://www.gao.gov/products/GAO-09-666]. Washington, D.C.: July 15, 
2009. 

Global HIV/AIDS: A More Country-Based Approach Could Improve 
Allocation of PEPFAR Funding. [hyperlink, 
http://www.gao.gov/products/GAO-08-480]. Washington, D.C.: April 2, 
2008. 

Global Health: Global Fund to Fight AIDS, TB and Malaria Has Improved 
Its Documentation of Funding Decisions but Needs Standardized 
Oversight Expectations and Assessments. [hyperlink, 
http://www.gao.gov/products/GAO-07-627]. Washington, D.C.: May 7, 2007. 

Global Health: Spending Requirement Presents Challenges for Allocating 
Prevention Funding under the President's Emergency Plan for AIDS 
Relief. [hyperlink, http://www.gao.gov/products/GAO-06-395]. 
Washington, D.C.: April 4, 2006. 

Global Health: The Global Fund to Fight AIDS, TB and Malaria Is 
Responding to Challenges but Needs Better Information and 
Documentation for Performance-Based Funding. [hyperlink, 
http://www.gao.gov/products/GAO-05-639]. Washington, D.C.: June 10, 
2005. 

Global HIV/AIDS Epidemic: Selection of Antiretroviral Medications 
Provided under U.S. Emergency Plan Is Limited. [hyperlink, 
http://www.gao.gov/products/GAO-05-133]. Washington, D.C.: January 11, 
2005. 

Global Health: U.S. AIDS Coordinator Addressing Some Key Challenges to 
Expanding Treatment, but Others Remain. [hyperlink, 
http://www.gao.gov/products/GAO-04-784]. Washington, D.C.: June 12, 
2004. 

Global Health: Global Fund to Fight AIDS, TB and Malaria Has Advanced 
in Key Areas, but Difficult Challenges Remain. [hyperlink, 
http://www.gao.gov/products/GAO-03-601]. Washington, D.C.: May 7, 2003. 

[End of section] 

Footnotes: 

[1] Congress first authorized PEPFAR in 2003. Pub. L. No. 108-25, § 
401, 117 Stat. 711, 745. 

[2] In this report, "treatment" refers to the services delivered to 
HIV-positive people who are receiving treatment with antiretroviral 
(ARV) drugs. "Care and treatment" refers to treatment plus a set of 
additional services provided to improve the quality of life for HIV-
infected people, including clinical, psychological, spiritual, social, 
and prevention services. 

[3] See UNAIDS, UNAIDS 2012 Global Report: UNAIDS Report on the Global 
AIDS Epidemic (Geneva, Switzerland: 2012). See also WHO, The Strategic 
Use of Antiretrovirals to Help End the HIV Epidemic (Geneva, 
Switzerland: 2012). 

[4] Pub. L. No. 110-293, 122 Stat. 2918. 

[5] For fiscal years 2009 through 2012, State, USAID, and HHS (CDC and 
the National Institutes of Health) allocated the following amounts for 
global HIV/AIDS programs: $6.5 billion, $6.6 billion, $6.5 billion, 
and $6.4 billion, respectively. For fiscal year 2013, these agencies 
requested $6.2 billion for global HIV/AIDS programs. Amounts include 
U.S. contributions to the Global Fund to Fight AIDS, Tuberculosis and 
Malaria. 

[6] See GAO, President's Emergency Plan for AIDS Relief: Per-Patient 
Costs Have Declined Substantially, but Better Cost Data Would Help 
Efforts to Expand Treatment, [hyperlink, 
http://www.gao.gov/products/GAO-13-345] (Washington, D.C.: March 15, 
2013). 

[7] In this report, "studies" refers to evaluations, assessments, and 
other studies of treatment programs and M&E systems in PEPFAR partner 
countries. 

[8] Other PEPFAR implementing agencies are the Departments of State, 
Defense, Labor, and Commerce and the Peace Corps. Additional HHS 
offices and agencies involved in PEPFAR implementation are the Office 
of Global Affairs, the Food and Drug Administration, the Health 
Resources and Services Administration, the National Institutes of 
Health, and the Substance Abuse and Mental Health Services 
Administration. 

[9] In this report, "PEPFAR country teams" refers to PEPFAR country 
and regional teams and their implementing partners collectively. 
PEPFAR implementing agencies obtain services for PEPFAR activities 
through grants, cooperative agreements, and contracts with selected 
implementing partners. These partners include U.S.-based 
nongovernmental organizations and host-country governmental and 
nongovernmental organizations. For more information, see GAO, 
President's Emergency Plan for AIDS Relief: Partner Selection and 
Oversight Follow Accepted Practices but Would Benefit from Enhanced 
Planning and Accountability, [hyperlink, 
http://www.gao.gov/products/GAO-09-666] (Washington, D.C.: July 15, 
2009). 

[10] The 33 countries were Angola, Botswana, Burundi, Cambodia, 
Cameroon, China, Côte d'Ivoire, the Democratic Republic of the Congo, 
the Dominican Republic, Ethiopia, Ghana, Guyana, Haiti, India, 
Indonesia, Kenya, Lesotho, Malawi, Mozambique, Namibia, Nigeria, 
Russia, Rwanda, South Africa, South Sudan, Swaziland, Tanzania, 
Thailand, Uganda, Ukraine, Vietnam, Zambia, and Zimbabwe. The three 
regions were the Caribbean, Central America, and Central Asia. 

[11] Pub. L. No. 110-293, § 301(c)(6). 

[12] According to OGAC guidance, partnership frameworks and their 
associated partnership framework implementation plans are not intended 
to be legally binding. Rather, they are intended as nonbinding joint 
strategic-planning documents that outline the collaborative 
relationship between the U.S. government and partner countries and 
reflect overarching 5-year goals and the commitments of each party. 
See U.S. Department of State, Office of the U.S. Global AIDS 
Coordinator, Guidance for PEPFAR Partnership Frameworks and 
Partnership Framework Implementation Plans, Version 2.0 (Sept. 14, 
2009), accessed January 28, 2013, [hyperlink, 
http://www.pepfar.gov/documents/organization/120510.pdf]. 

[13] As of February 2013, partnership frameworks had been signed with 
the following 22 partner countries and regions: Angola, Botswana, the 
Caribbean region, the Central American region, the Democratic Republic 
of the Congo, the Dominican Republic, Ethiopia, Ghana, Haiti, Kenya, 
Lesotho, Malawi, Mozambique, Namibia, Nigeria, Rwanda, South Africa, 
Swaziland, Tanzania, Ukraine, Vietnam, and Zambia. 

[14] U.S. Department of State, Office of the U.S. Global AIDS 
Coordinator, PEPFAR Blueprint: Creating an AIDS-free Generation 
(Washington, D.C.: November 2012). 

[15] This is typically measured by CD4 (cluster of differentiation 
antigen 4) count in a sample of blood. CD4 cells are a type of white 
blood cell that fights infection. Along with other tests, the CD4 
count helps determine the strength of the immune system, indicates the 
stage of the HIV disease, guides treatment, and predicts the disease's 
progress. 

[16] WHO recommended treatment for all people with CD4 counts of less 
than 350 cells/mm3. Prior to 2010, WHO's guidelines recommended 
treatment for all people with CD4 counts of less than 200 cells/mm3. 
Normal CD4 counts range from 500-1,000 cells/mm3. 

[17] In some cases, this may be done through laboratory assessments of 
the severity of the patient's HIV infection, as measured by viral 
load. Viral load is a measure of the severity of a viral infection and 
can be calculated by estimating the amount of virus in an involved 
body fluid. 

[18] See GAO, President's Emergency Plan for AIDS Relief: Program 
Planning and Reporting, [hyperlink, 
http://www.gao.gov/products/GAO-11-785] (Washington, D.C.: July 29, 
2011). 

[19] Office of the U.S. Global AIDS Coordinator, The President's 
Emergency Plan for AIDS Relief, Next Generation Indicators Reference 
Guide: Planning and Reporting, Version 1.1 (August 2009), accessed 
March 20, 2013, [hyperlink, 
http://www.pepfar.gov/documents/organization/81097.pdf. The guidance 
went into effect for the fiscal year 2010 planning and reporting cycle 
and was updated in February 2013. 

[20] According to OGAC, 23 PEPFAR country teams provided direct 
support for treatment services in fiscal years 2011 and 2012: 
Botswana, Cambodia, China, Côte d'Ivoire, the Democratic Republic of 
the Congo, the Dominican Republic, Ethiopia, Guyana, Haiti, India, 
Kenya, Lesotho, Mozambique, Namibia, Nigeria, Rwanda, South Africa, 
Swaziland, Tanzania, Uganda, Vietnam, Zambia, and Zimbabwe. OGAC 
provides guidance to country teams for determining whether people 
receive direct services through PEPFAR implementing partners. See 
appendix II for more information. 

[21] OGAC also requires PEPFAR country teams to collect and maintain 
data on two additional treatment-related indicators: (1) percentage of 
health facilities offering treatment services and (2) percentage of 
health facilities providing treatment services using CD4 monitoring in 
line with national guidelines, on site, or through referral. These 
data are not reported to OGAC. 

[22] These requirements include a plan to increase the number of 
people receiving treatment, annual reporting requirements, and 
PEPFAR's annual treatment target. Pub. L. No. 110-293, §§ 101, 301, 
403. 

[23] In addition to tracking the number of people currently on 
treatment, in fiscal year 2010, OGAC also began tracking the number of 
people newly enrolled in treatment that is directly provided by 
PEPFAR. Data reported for this indicator--which OGAC disaggregates by 
age, sex, and pregnancy status for women--provide additional 
information on treatment program expansion and links treatment 
programs to special efforts aimed at preventing mother-to-child 
transmission of HIV. 

[24] PEPFAR country teams also report data disaggregated by age, sex, 
and pregnancy status for women. 

[25] In December 2011, the President announced an increase in PEPFAR's 
5-year target for the number of people receiving treatment supported 
directly by PEPFAR--from 4 million to 6 million by the end of fiscal 
year 2013. 

[26] The number of people on treatment directly provided by PEPFAR 
includes an estimate of the results of treatment programs supported by 
the Global Fund to Fight AIDS, Tuberculosis and Malaria (Global Fund). 
To estimate the overlap, OGAC and the Global Fund together analyze 
available treatment program data, taking into account the contribution 
each makes to partner countries' treatment programs to determine 
likely overlap. For example, for fiscal year 2010, OGAC reported that 
PEPFAR had directly provided treatment for about 3.2 million people, 
which includes an estimated overlap with the Global Fund of about 1.5 
million. 

[27] A recent version of the guidance, dated February 2013, does not 
include this statement. In its technical comments, OGAC indicated that 
this statement reflected initial thinking about alternative methods 
for representing PEPFAR's contributions to partner countries' HIV/AIDS 
programs. 

[28] This percentage is calculated by dividing the number of adults 
and children currently receiving treatment by an estimate (typically 
derived from epidemiological data and modeling) of the number of 
adults and children eligible to receive treatment (i.e., those with 
advanced HIV infection). Population coverage indicators generally 
depict regional or national program results and are to be 
disaggregated by age and sex. 

[29] The most recent available information on national treatment 
coverage are from UNAIDS. We consulted UNAIDS's most recent report on 
the global AIDS epidemic and cross-referenced this information with 
national treatment coverage data available on the UNAIDS website as 
well data in PEPFAR country teams' fiscal year 2011 annual reports to 
OGAC. Data on China's 2011 national coverage rate were not available 
from UNAIDS. In its fiscal year 2011 report to OGAC, the PEPFAR 
country team provided information on China's 2010 national treatment 
coverage rate, which we used for this report. 

[30] Each country sets its own targets for national treatment coverage 
rates. According to UNAIDS, most countries have set 80 percent as 
their target for treatment coverage. 

[31] See World Health Organization, Retention in HIV Programmes: 
Defining the Challenges and Identifying Solutions (Geneva, 
Switzerland: 2011). 

[32] Patient monitoring is the routine collection, compilation, and 
analysis of patient data over time and across service delivery points, 
using information from paper forms or entered into a computer. Patient 
management--also referred to as clinical management or clinical 
monitoring--is the relationship between health providers and patients 
over time, assisted by written records. 

[33] The process of routinely tracking information about a program and 
its outcomes is referred to as program monitoring. Program monitoring 
at facility, district, and national levels requires many types of 
information, including aggregated patient data. 

[34] As of March 2013, partnership frameworks had been signed with the 
following 22 partner countries and regions: Angola, Botswana, 
Caribbean region, Central American region, Democratic Republic of the 
Congo, Dominican Republic, Ethiopia, Ghana, Haiti, Kenya, Lesotho, 
Malawi, Mozambique, Namibia, Nigeria, Rwanda, South Africa, Swaziland, 
Tanzania, Ukraine, Vietnam, and Zambia. 

[35] In addition, OGAC technical guidance cites an assessment tool 
established by UNAIDS, with PEPFAR support, which partner countries 
can use to develop their M&E systems for HIV/AIDS programs. See 
UNAIDS, Organizing Framework for a Functional National HIV Monitoring 
and Evaluation System (Geneva: April 2008). 

[36] In this report, "PEPFAR country teams" refers to PEPFAR country 
and regional teams and their implementing partners collectively. 

[37] GAO, Tax Administration: IRS Needs to Further Refine Its Tax 
Filing Season Performance Measures, [hyperlink, 
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 
2002). 

[38] As of February 2013, the United States had signed partnership 
frameworks with the following 22 partner countries and regions: 
Angola, Botswana, the Caribbean region, the Central American region, 
the Democratic Republic of the Congo, the Dominican Republic, 
Ethiopia, Ghana, Haiti, Kenya, Lesotho, Malawi, Mozambique, Namibia, 
Nigeria, Rwanda, South Africa, Swaziland, Tanzania, Ukraine, Vietnam, 
and Zambia. 

[39] See GAO, President's Emergency Plan for AIDS Relief: Program 
Planning and Reporting, [hyperlink, 
http://www.gao.gov/products/GAO-11-785] (Washington, D.C.: July 29, 
2011). 

[40] In this report, "studies" refers to evaluations, assessments, and 
other studies of treatment programs and M&E systems in PEPFAR 
countries. 

[41] OGAC defines treatment retention as the percentage of adults and 
children known to be alive and on treatment 12 months after treatment 
initiation. 

[42] Patient loss-to-follow-up is defined as patients with unknown 
treatment outcomes, including possible death, treatment cessation, or 
self-transfer to another treatment facility. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]