Foreign Assistance: Federal Monitoring and Evaluation Guidelines Incorporate Most but Not All Leading Practices
Fast Facts
Are federal agencies doing a good job keeping track of the effectiveness of the foreign assistance programs they administer?
We looked at how agencies monitor and evaluate these programs. Specifically, we looked at the guidelines the Office of Management and Budget established for federal agencies with foreign assistance programs. We reviewed whether the guidelines incorporated leading practices for monitoring and evaluation, and whether agencies adopted the guidelines.
We found the OMB guidelines included 23 of 28 leading practices, and most agencies adopted most of them. We made 7 recommendations to increase the use of OMB's guidelines.
[Image updated to show examples of country flags.]
Photo of flags
Highlights
What GAO Found
The Office of Management and Budget's (OMB) foreign assistance Guidelines incorporate most of GAO's leading practices for monitoring and evaluation (M&E), but gaps exist (see figure).
Summary of Office of Management and Budget's Guidelines Addressing GAO's Leading Practices for Monitoring and Evaluation
- Monitoring: The Guidelines define monitoring as the continuous tracking of program or project data to determine whether desired results are as expected during implementation. The Guidelines do not require GAO's leading practices on risk assessments, staff qualifications, and program close-out procedures.
- Evaluation: The Guidelines define evaluation as the systematic collection and analysis of program or project outcomes for making judgments and informing decisions. They do not require GAO's leading practices on developing staff skills and following up on recommendations.
OMB officials indicated the Guidelines are focused on elements required in the Foreign Aid Transparency and Accountability Act of 2016 (FATAA), but noted that agencies can add additional requirements to their own M&E policies. FATAA requires the President to set forth guidelines “according to best practices of monitoring and evaluation.” OMB staff acknowledged that GAO's leading practices are important, but stated that there is no singular established standard for best monitoring practices. Nevertheless, all of GAO's leading practices can help agencies address impediments, effectively manage foreign assistance, and meet their goals.
When assessing agencies' M&E policies against OMB Guidelines, GAO found that agencies incorporated most of the requirements. However, for monitoring, one of the six agencies GAO reviewed—DOD—did not include the requirements to establish agencies' roles and responsibilities and ensure verifiable data for monitoring activities. For evaluation, agencies required most Guideline requirements, but not all. For example, DOD, HHS, and USDA did not require conducting impact evaluations for pilot programs or projects. Without a clear requirement to do such evaluations, agencies risk duplicating or scaling up programs without fully understanding the factors that could lead to their success or failure. Agencies GAO reviewed have plans or mechanisms in place to oversee the implementation of their M&E policies. For example, State developed a guidance document to operationalize and oversee its M&E policy to ensure the implementation of the Guidelines.
Why GAO Did This Study
The Trump Administration requested $28.5 billion in foreign assistance in fiscal year 2019, to be administered by at least 22 federal agencies. Almost 95 percent of this assistance is administered by six agencies—the Departments of Agriculture (USDA), Defense (DOD), State (State), Health and Human Services (HHS), the Millennium Challenge Corporation (MCC), and the U.S. Agency for International Development (USAID). FATAA required the President to set forth guidelines for M&E of U.S. foreign assistance. In January 2018, OMB issued the required guidelines for federal agencies. FATAA also contained a provision for GAO to analyze the guidelines established by OMB; and assess the implementation of the guidelines by the agencies.
In this report, GAO examined the extent to which (1) OMB's M&E Guidelines incorporate GAO leading practices, and (2) agencies incorporate the OMB Guidelines in their M&E policies and plans. GAO assessed the OMB Guidelines against GAO's 28 leading practices identified in GAO-16-861R . GAO also assessed the six agencies' foreign assistance M&E policies against the Guidelines and interviewed OMB and relevant agency officials in Washington, DC.
Recommendations
GAO is making recommendations to OMB, DOD, State, and USDA. OMB did not agree with the recommendation to update the Guidelines, but GAO maintains that doing so can help to emphasize the importance of the M&E practices we identified. DOD, State, and USDA agreed with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget |
Priority Rec.
The Director of the Office of Management of Budget should update the Guidelines to include GAO's leading practices of developing monitoring plans that are based on risks, ensuring that monitoring staff have appropriate qualifications, establishing procedures to close-out programs, developing staff skills regarding evaluations, and establishing mechanisms for following up on evaluation recommendations. (Recommendation 1) |
OMB disagreed with the recommendation and suggested it would be more effective to remind agencies that, in addition to the guidelines, they should follow all other relevant OMB guidance affecting monitoring and evaluation. OMB asserted that this guidance contains provisions relevant to our leading practices not included in the Foreign Assistance Monitoring and Evaluation Guidelines. However, we continue to believe that it is important for OMB to incorporate this other guidance into the Foreign Assistance Monitoring and Evaluation Guidelines, if only by reference, to emphasize the importance of these practices in the context of monitoring and evaluation of foreign assistance. As of February 2024, OMB had not taken any actions to address this recommendation and has indicated that it does not plan to take any action, but GAO will continue to monitor this issue.
|
| Department of Defense | The Secretary of Defense should update the Department's monitoring and evaluation policies to define roles and responsibilities among agencies that participate in interagency funding transfers. (Recommendation 2) |
DOD partially concurred with this recommendation and noted that DOD has not utilized the authority provided by 10 U.S.C. 385, which allows DOD to transfer funds to other departments and agencies of the U.S. government for the purposes of implementing or supporting foreign assistance programs. However, DOD indicated it would implement our recommendation to define roles and responsibilities among agencies that participate in interagency funding transfers, should such transfers become necessary. DOD provided updated guidance, the Standards and Guidelines for Security Cooperation Performance Monitoring and the Standards and Guidelines for Security Cooperation Independent and Self-Evaluation, in June 2020 , which specify that DOD will work with relevant parties to consider and document roles and responsibilities for monitoring and evaluation of funded work and outline information sharing processes in the event that DOD transfers funding for security cooperation assistance to another agency .
|
| Department of Defense | The Secretary of Defense should update the Department's monitoring and evaluation policies to ensure verifiable, reliable, and timely data are available to monitoring personnel. (Recommendation 3) |
DOD concurred with the recommendation and provided the updated guidance, Standards and Guidelines for Security Cooperation Performance Monitoring, in June 2020. This updated guidance specifies that monitoring frameworks should employ verifiable, reliable, and timely quantitative and qualitative data in accordance with OMB requirements. The updated guidelines also define these terms, which may provide additional clarity for officials.
|
| Department of Defense | The Secretary of Defense should update the Department's monitoring and evaluation policies to ensure that it evaluates all programs, at least once in their lifetimes, whose dollar value equals or exceeds that of the median program in the agency. (Recommendation 4) |
DOD concurred with this recommendation and provided the updated guidance, Standards and Guidelines for Security Cooperation Independent and Self-Evaluation, in June 2020. The updated guidelines specify that DOD will evaluate all initiatives whose dollar value equals or exceeds the median for the relevant account at least once in their lifetime, in accordance with OMB and Congressional requirements.
|
| Department of Defense | The Secretary of Defense should update the Department's monitoring and evaluation policies to require the agency to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 5) |
DOD concurred with the recommendation and provided updated guidance, its Standards and Guidelines for Security Cooperation Independent and Self-Evaluation, in June 2020. The updated guidelines state that DOD may conduct impact evaluations, as feasible and appropriate, or will provide a justification for conducting a different type of evaluation. According to DOD officials, the updated guidance regarding impact evaluations covers all types of security cooperation initiatives, including pilot programs. DOD guidance already stipulates that pilot programs should be evaluated and the updated guidance clarifies that DOD must justify conducting an evaluation other than an impact evaluation. The updated guidance meets the intent of the recommendation for DOD to establish a policy that helps to ensure that impact evaluations for its pilot assistance programs are conducted or that a justification for not conducting an impact evaluation is provided.
|
| Department of State | The Department of State's U.S. Global AIDS Coordinator, in collaboration with HHS and other implementing agencies, should update the PEPFAR monitoring and evaluation policies to require these agencies to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 6) |
State agreed with the intent of the recommendation and updated its PEPFAR guidance in February 2021. The updated guidance states that impact evaluations (as defined by OMB) are often impractical in the context of PEPFAR. However, the guidance stipulates that PEPFAR operating units can propose pilot programs and associated impact evaluations for consideration during the Country and Regional Operational planning process. The guidance advises, though, that operating units consider whether alternative methods of monitoring, evaluation, or research are justifiably sufficient to assess the effectiveness of a proposed pilot program. The updated guidance meets the intent of the recommendation for State to establish a policy that helps to ensure that impact evaluations for its pilot assistance programs are conducted or that a justification for not conducting an impact evaluation is provided, which will ensure that agencies do not duplicate or scale up programs without fully understanding the factors that could lead to their success or failure.
|
| Department of Agriculture | The Secretary of Agriculture, in collaboration with the Foreign Agriculture Service, should update their monitoring and evaluation policies to require USDA to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 7) |
USDA agreed with the recommendation and updated its Evaluation Plan Checklist in August 2019. The updated checklist specifies that a justification must be provided in the evaluation plan if the implementer is not proposing to conduct an impact evaluation for a project. This update meets the intent of the recommendation for USDA management to establish a policy that helps to ensure that impact evaluations for its pilot assistance programs are conducted when appropriate and feasible.
|