Millennium Challenge Corporation:

Independent Reviews and Consistent Approaches Will Strengthen Projections of Program Impact

GAO-08-730: Published: Jun 17, 2008. Publicly Released: Jun 17, 2008.

Additional Materials:

Contact:

David B. Gootnick
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In January 2004, Congress established the Millennium Challenge Corporation (MCC) for foreign assistance. Eligible countries submit compact proposals for MCC funding for projects aimed at reducing poverty through economic growth. To assess the proposed compacts' likely impact, MCC performs economic analyses estimating the compacts' economic rate of return (ERR) and effects on income and poverty as well as the number of compact beneficiaries. MCC uses these analyses to inform its decisions to fund proposed compacts and to inform Congress and the public about its progress in achieving its mission of poverty reduction through economic growth. GAO was asked to examine MCC's projections of (1) ERR and (2) compacts' impact on income and poverty as well as numbers of beneficiaries. GAO reviewed MCC's stated impacts and analyses for four MCC compacts that represented 41 percent of MCC's compact assistance and met with MCC officials.

MCC used different time frames and methods to calculate ERRs for its compacts with Armenia, El Salvador, Lesotho, and Mozambique. In calculating ERR for 20 projects within the compacts, MCC used a 20-year time frame for 9 projects and used different time frames for the other 11. In 2 of the 11 projects, using a 20-year time frame, as MCC used for similar projects, would reduce the ERR below the level MCC set as the minimum acceptable ERR. At the compact level, MCC's use of varying time frames did not affect the ERRs significantly. MCC used varying methods to account for the costs of same-sector projects, although its approaches to determining project benefits were generally similar. MCC also used two different methods to calculate compact-level ERR; however, the choice of method did not reduce it below the minimum ERR. In three of the four compacts that we reviewed, MCC did not retain documentation of the economic analyses used to support the investment decision, but continued to modify the analyses. MCC has recently begun to standardize elements of its economic analyses and centralize its records management. MCC identified and corrected analytic errors in its projections of impact on income and poverty. MCC also used varying methods to project these impacts. In responding to GAO's questions about its published projections of impact on income and poverty, MCC identified analytic errors for three of the four compacts and, in correcting these errors, generally lowered the projected impacts. Correcting these errors raised one projection by 5 percent but reduced others by 3 percent to 96 percent. According to MCC officials, the revised projections would not have affected MCC's decision to recommend signing the compacts. The officials noted that, in the future, compact impact projections will undergo peer review. However, MCC has not documented procedures for these reviews. MCC used varying methods for its projections of impact on income and poverty, limiting comparability and replicability across compacts. To project impact on income for Armenia, El Salvador, and Mozambique, MCC estimated the compacts' impact by summing the total benefits of individual compact projects and adding them to the income that would have prevailed without MCC. However, for Lesotho, MCC estimated the impact on income based on the published results of a World Bank model based on elements different from those of the MCC compact. In response to our questions, MCC revised its initial estimate of the effect of income growth on poverty for Mozambique by presenting two estimates, based on Mozambique-specific and cross-country data, respectively. Although a number of methods for projecting poverty impact are valid, the method chosen can affect the results, and MCC's guidelines do not identify preferred methods for these calculations. MCC also used varying methods to estimate numbers of beneficiaries for the compacts and has not provided specific criteria for defining beneficiaries; however, MCC officials reported they are taking steps to provide more detailed guidance.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To improve the reliability and comparability of its projected ERR and economic impacts, the CEO of MCC should adopt and implement written procedures for a secondary independent review of the methods and results of its economic analyses.

    Agency Affected: Millennium Challenge Corporation

    Status: Closed - Implemented

    Comments: In response to the GAO recommendation, the Millennium Challenge Corporation (MCC) adopted an internal peer review process coordinated by MCC's Chief Economist. MCC has used this new process on the Burkina Faso, Namibia, and Senegal compact proposals. In addition, MCC has posted its economic analysis spreadsheets for compact countries online to make them available for review outside of MCC.

    Recommendation: To improve the reliability and comparability of its projected ERR and economic impacts, the CEO of MCC should improve MCC's guidelines by identifying a consistent approach with preferred methods for projecting compacts' impact on income and poverty.

    Agency Affected: Millennium Challenge Corporation

    Status: Closed - Implemented

    Comments: The Millennium Challenge Corporation (MCC) has adopted revised Guidelines on Economic and Beneficiary Analysis that include specific guidance on counting beneficiaries, estimating the incidence of benefits, and disaggregating benefits by demographic groups.

    Jul 24, 2014

    Jul 21, 2014

    Jul 9, 2014

    Jul 8, 2014

    Jun 26, 2014

    Jun 25, 2014

    Looking for more? Browse all our products here