Public Financial Management: Improvements Needed in USAID's and Treasury's Monitoring and Evaluation Efforts
Highlights
What GAO Found
To develop programs to strengthen developing countries’ Public Financial Management (PFM) systems, the U.S. Agency for International Development (USAID) and the U.S. Department of the Treasury (Treasury) rely on assessments of the host country government’s systems. In 2011, USAID implemented new processes that place a greater emphasis on PFM in its development efforts as the agency aims to increase its use of country systems to deliver assistance. The agency traditionally included PFM capacity-building efforts only as components of broader programs, as it identified relevant weaknesses during the country assessment or program design process. USAID’s new strategy and program development processes include a mandatory assessment of a country’s institutional capacity, including its financial systems, and a requirement to consider the use of country systems to deliver assistance. Most USAID country offices are required to develop a strategy using the new guidance by the end of fiscal year 2013. Treasury’s process for developing programs begins with an initial assessment of the host country’s capabilities. Treasury staff then draft objectives for the program. For example, a Treasury program in Honduras set four objectives, including improving operational efficiency and enhancing accountability by strengthening the organization of the ministry of finance. Once in country, the advisor develops an annual workplan, outlining more specific goals aimed at meeting the overall objectives.
USAID and Treasury use several processes to monitor and evaluate their PFM assistance, but weaknesses exist. USAID uses its regular procedures, which may include performance management plans, periodic progress reporting, site visits, and evaluations, to monitor and evaluate its PFM-related programs. Prior reports by USAID’s Inspector General and GAO have found weaknesses in USAID’s implementation of its monitoring procedures in other programs, including programs from the USAID offices that provide PFM assistance. In addition, USAID is currently unable to monitor overall progress toward its target to obligate 30 percent of its program funds through local systems by 2015. USAID, and GAO in prior reports, have identified a number of weaknesses in evaluation practice. To address weaknesses the agency had identified, USAID adopted a new evaluation policy in January 2011 that states that all large projects are required to have an external evaluation, 3 percent of program budgets should be devoted to external evaluation, and evaluations must use methods that generate the highest quality evidence. Treasury’s processes for monitoring and evaluating its programs include monthly reports, annual quantitative performance measures, voluntary customer feedback surveys, and on-site management reviews, but Treasury does not fully evaluate the performance of its completed technical assistance programs. In addition, Treasury’s quantitative performance measures have been a useful project-level indicator of performance but have not been a useful indicator of overall performance due in part to inherent challenges associated with summarizing program performance and errors introduced when aggregating the performance data. Furthermore, a senior Treasury official reported that Treasury had not yet fully implemented a requirement to conduct independent postproject evaluations of its technical assistance programs.
Why GAO Did This Study
Effective management of public resources can play an important role in a country’s development. In recent years, developing countries committed to strengthen their PFM systems and donors committed to use those systems as much as possible. The United States provides assistance to strengthen PFM systems primarily through USAID and Treasury. USAID conducts capacity building activities to strengthen PFM systems as part of its development programs and has also set a target to obligate 30 percent of its annual assistance through local systems by 2015. Treasury provides technical assistance through advisors who work in country, typically with the finance ministry.
GAO was asked to examine the processes U.S. agencies use to (1) develop programs to strengthen PFM systems and (2) monitor and evaluate those programs. GAO reviewed agency guidance and program documents, interviewed U.S. agency officials, and selected case studies to serve as illustrative examples of PFM-related programs.
Recommendations
USAID should improve its capacity to measure its use of local systems and ensure adequate monitoring of its PFM programs. Treasury should implement additional controls to improve the process for computing program-wide annual performance measures and fully implement its requirement to evaluate the impact of its completed assistance. USAID and Treasury both concurred with GAOs recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| U.S. Agency for International Development | To monitor progress toward USAID's target to obligate 30 percent of its annual assistance through local systems by 2015, the Administrator of USAID should direct the appropriate offices to develop a process to reliably identify and track the agency's use of local systems in all countries receiving assistance. |
USAID agreed with GAO's recommendation and in response, developed a process to track progress of providing assistance through local systems, as part of a broader effort to track progress of all USAID forward reform components. USAID first reported progress on the use of local systems in the 2013 USAID Forward Progress Report.
|
| U.S. Agency for International Development | To help ensure that USAID conducts effective evaluations of PFM-related programs under its new policy, the Administrator of USAID should direct the appropriate offices to ensure that they are establishing adequate monitoring practices for PFM-related programs. Such practices may include selecting proper indicators, collecting reliable baseline data, and ensuring the reliability of reported results. |
USAID agreed with GAO's recommendation and in response, issued the revised Guide to Public Financial Management in April 2015, which provides guidelines as to what activities/projects should be planned and in what sequence. The Guide also provides guidance on how to select indicators for the Performance Monitoring Plan for the monitoring and evaluation of the strengthening public financial management activity or project and how Missions might use the indicators contained in the Guide. This guidance, including the updated information on indicator selection, was integrated into the Agency's Strengthening Public Financial Management and Public Accountability course, first taught in June 2015.
|
| Department of the Treasury | To improve the effectiveness of OTA's technical assistance, the Secretary of the Treasury should direct the Office of Technical Assistance (OTA) to implement additional controls to improve the process for computing OTA-wide annual performance measures. |
In response to the draft of the report provided to OTA, OTA commented that they had already implemented this recommendation by: correcting transcription errors; taking steps to improve data quality controls to prevent transcription errors; improving formatting and usability of the data collection instrument; and increasing staff resources to these efforts. Since the release of the report, OTA reported that in response to the GAO recommendation, OTA has corrected the transcription and arithmetic errors identified in the annual performance measurement exercise, as well as taking steps to strengthen internal controls. OTA's data collection instrument and spreadsheet used to aggregate data now include fields that automatically calculate averages. According to an OTA official, additional internal control measures include an Associate Director's review of the data submitted through the data collection instrument and separate spot checks of the aggregated data set.
|
| Department of the Treasury | To improve the effectiveness of OTA's technical assistance, the Secretary of the Treasury should direct the Office of Technical Assistance (OTA) to fully implement OTA's existing requirement for end-of-project evaluations and, consistent with its existing guidance, have an independent party conduct the evaluations to improve monitoring and evaluation. |
OTA fully concurred with this recommendation and in response to this report, began full implementation of this requirement. Since December 2012, OTA has completed 17 end-of-project reports, bringing the program completely up to date for each project concluded since the requirement was instated in December 2009. In addition, an independent party--someone other than the project implementer--conducts the evaluation. According to OTA officials, the evaluations are undertaken typically three to six months following the end of OTA's assistance activities to better understand the program's longer-term impact.
|