This is the accessible text file for GAO report number GAO-11-544 entitled 'International School Feeding: USDA's Oversight of the McGovern-Dole Food for Education Program Needs Improvement' which was released on May 19, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: May 2011: International School Feeding: USDA's Oversight of the McGovern-Dole Food for Education Program Needs Improvement: GAO-11-544: GAO Highlights: Highlights of GAO-11-544, a report to congressional requesters. Why GAO Did This Study: The McGovern-Dole International Food for Education and Child Nutrition Program (MGD Program) provides donations of U.S. agricultural products and financial and technical assistance for school feeding programs in the developing world. According to the U.S. Department of Agriculture (USDA), with about $200 million in funding in fiscal year 2010, the MGD Program served about 5 million beneficiaries in 28 countries. In 2006 and 2007, USDA’s Office of the Inspector General (OIG) audited the department’s food aid programs and identified significant weaknesses. This report examines (1) USDA’s oversight of the MGD Program and (2) the extent to which USDA has addressed the program’s internal control weaknesses. GAO conducted field work in Cambodia, Guatemala, and Kenya; reviewed USDA and implementing partners’ documents and studies on school feeding; and interviewed officials from U.S. agencies and various organizations. What GAO Found: USDA has established a grant approval process consistent with the MGD Program’s objectives of education, nutrition, and sustainability, but weaknesses in its oversight limit its ability to ensure that the program’s objectives are met. Specifically, USDA has established criteria for designating priority countries, assessing proposals, and negotiating grant agreements with the United Nations World Food Program and nongovernmental organizations that implement the MGD Program. In addition to providing in-school meals and take-home rations, USDA supports complementary activities such as teacher training, nutrition education, and fostering parental involvement. The oversight weaknesses that GAO identified include: * USDA provides weak performance monitoring of the MGD Program’s implementation. For example, USDA does not systematically analyze implementing partners’ reporting and provides limited feedback. In addition, requirements for implementing partners do not ensure consistent reporting and lack performance indicators directly measuring educational progress, such as learning, and, in some cases, nutrition. GAO’s Standards for Internal Control in the Federal Government states that activities need to be established to monitor performance measures and indicators and that these controls could call for comparisons and assessments relating different sets of data to one another so that analysis of the relationships can be made and appropriate actions taken. USDA does not conduct systematic site visits and relies on the implementing partners’ performance monitoring, whose rigor varies by implementing partner. * USDA has not evaluated completed MGD projects, but is taking steps that will emphasize evaluation in the future. Although USDA now requires implementing partners to conduct evaluations, it has not yet established policies and procedures to guide these evaluations. The American Evaluation Association’s An Evaluation Roadmap for a More Effective Government recommends that agencies develop policies and procedures to guide evaluation and assess the strengths and weaknesses of programs to improve their effectiveness. GAO found that USDA is taking steps to improve its financial oversight of MGD projects, but further improvements would help strengthen its internal controls. GAO’s Standards for Internal Control in the Federal Government also states that managers need to compare actual performance to planned results and analyze significant differences in a timely manner. USDA has controls in place over project expenditures, but a lack of timely grant closeouts prevents USDA from ensuring that grantees have met all financial requirements and that unused or misused funds are promptly reimbursed to USDA. To date, USDA has collected over $850,000 in unused or misused funds via grant closeouts; however, 15 of 42 (36 percent) of the MGD grants that are eligible to be closed, remain open. Finally, USDA has related outstanding OIG audit recommendations to implement and close. What GAO Recommends: GAO recommends that the Secretary of Agriculture improve oversight of the MGD Program, including issuing monitoring and evaluation guidance for implementing partners, and formalize policies and procedures for closing out grant agreements and establishing guidance to determine when agreements should be closed. USDA agreed with GAO’s recommendations and said that it will take steps to address them. View [hyperlink, http://www.gao.gov/products/GAO-11-544] or key components. For more information, contact Tom Melito at (202) 512-9601 or melitot@gao.gov. [End of section] Contents: Letter: Background: USDA Has Established a Grant Approval Process Consistent with the MGD Program's Objectives, but Oversight Weaknesses Make It Difficult to Determine the Extent to which the Program Is Achieving Its Objectives: USDA Has Taken Actions to Address the MGD Program's Internal Control Weaknesses, but Improvements Can Be Made: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objective, Scope, and Methodology: Appendix II: GAO Analysis of Selected Studies on School Feeding Programs: Appendix III: Operation of the MGD Program: Appendix IV: MGD Program Funding by Year, Country, and Region: Appendix V: Methods NGOs Planned to Use to Select MGD Program Beneficiaries: Appendix VI: MGD Program's Planned Complementary Activities in Agreements with Implementing Partners: Appendix VII: MGD Program's Performance Indicators in NGOs Agreements and WFP Proposals: Appendix VIII: Comments from the U.S. Department of Agriculture: GAO Comments: Appendix IX: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: MGD Program Performance Indicators in NGOs Agreements and WFP Proposals: Table 2: Analyses of Progress Reports Submitted by Implementing Partners in Our Fieldwork Countries: Table 3: Results of the MGD Program Financial Compliance Reviews as Reported by USDA: Table 4: Summary of Selected Studies on School Feeding Programs: Table 5: Criteria Used to Determine Priority Countries: Table 6: Criteria Used to Assess Proposals, Fiscal Years 2008-2010: Table 7: Periodic Progress Reporting Required of Implementing Partners: Table 8: MGD Program Obligations by Year, Country, and Region, Fiscal Years 2003 through 2010 (in millions): Table 9: Methods NGOs Planned to Use to Select MGD Program Beneficiaries: Table 10: MGD Program's Planned Complementary Activities in Agreements with Implementing Partners: Table 11: MGD Program's Performance Indicators in NGOs Agreements and WFP Proposals: Figures: Figure 1: Evolution of the MGD Program: Figure 2: MGD Program Countries Since 2003: Figure 3: Key Elements of the Grant Approval, Monitoring, and Evaluation Phases: Figure 4: The Grant Approval Phase: Figure 5: Examples of Complementary Activities in Agreements with Implementing Partners: Figure 6: The Monitoring Phase: Figure 7: Number of MGD Countries Visited and Not Visited, Fiscal Years 2004-2010: Figure 8: The Evaluation Phase: Figure 9: MGD Program Funds Obligated by Region, Fiscal Years 2003 Through 2010: Abbreviations: AEA: American Evaluation Association: Compliance: Compliance, Security and Emergency Planning Division: FAD: Food Assistance Division: FAIS: Food Aid Information System: FAS: Foreign Agricultural Service: MES: Monitoring and Evaluation Staff: MGD Program: McGovern-Dole International Food for Education and Child Nutrition Program: NGO: nongovernmental organization: OCFO: Office of the Chief Financial Officer: OIG: Office of the Inspector General: ROM: results-oriented management: UNICEF: United Nations Children's Fund: USDA: U.S. Department of Agriculture: USAID: U.S. Agency for International Development: WFP: World Food Program: [End of section] United States Government Accountability Office: Washington, DC 20548: May 19, 2011: Congressional Requesters: In 2009, the United Nations World Food Program (WFP)[Footnote 1] estimated that about 66 million children in developing countries attend school hungry. The McGovern-Dole International Food for Education and Child Nutrition Program (MGD Program) was established by Congress in 2002 to help address this problem by providing donations of U.S. agricultural products, as well as financial and technical assistance, for school feeding and maternal and child nutrition projects in low-income countries to achieve improvements in education, nutrition, and sustainability of school feeding. According to the U.S. Department of Agriculture (USDA), with funding of about $200 million in fiscal year 2010, the MGD Program served approximately 5 million beneficiaries in 28 countries. The program is administered by USDA's Foreign Agricultural Service (FAS) and implemented by WFP and nongovernmental organizations (NGO). In 2002, we identified a number of weaknesses with the pilot program that preceded the MGD Program,[Footnote 2] such as a lack of expertise and staff resources dedicated to school feeding and a lack of performance indicators to monitor and evaluate project results. In 2006 and 2007, USDA's Office of the Inspector General (OIG) audits of FAS food aid programs, including the MGD Program, identified significant weaknesses, including the lack of a strategic planning process for food aid programs, lack of outcome-based performance measures, and poor controls over on-site visits to implementing partner projects. As part of our work on international food assistance,[Footnote 3] you asked us to review the MGD Program. In this report we assessed (1) the extent to which USDA is overseeing the MGD Program to ensure it is achieving its objectives, and (2) the extent to which USDA has addressed the MGD Program's internal control weaknesses. To address these objectives, we reviewed USDA and implementing partners' program documents and financial information as well as numerous studies of school feeding programs. We conducted fieldwork in Cambodia, Guatemala, and Kenya and met with representatives of WFP and NGOs that implement the program. We also met with staff from U.S. missions, host governments, the World Bank, the United Nations Children's Fund (UNICEF), and the United Nations Food and Agriculture Organization in these countries as appropriate. We visited selected schools at which MGD projects are implemented in each country as well as food warehouses. In Washington, D.C., we interviewed officials from U.S. agencies, including USDA and the U.S. Agency for International Development (USAID), as well as private foundations and research institutions, such as the Bill and Melinda Gates Foundation, the Delhi School of Economics, the Global Child Nutrition Foundation, and International Food Policy Research Institute. In Rome, we met with representatives of the U.S. Mission to the United Nations Food and Agriculture Agencies, WFP, the Food and Agriculture Organization, and several bilateral donors. We analyzed USDA's grant approval process, including NGOs' grant agreements with USDA, as well as WFP proposals and country programs, initiated in fiscal years 2008 through 2010 to determine whether the process is consistent with the goals of the MGD Program. In addition, we reviewed the NGOs' and WFP's progress reports submitted to USDA for the countries we visited to obtain illustrative examples of what they reported about their school feeding projects. We also reviewed audit reports by the USDA OIG, which highlighted relevant internal control weaknesses. We then reviewed various reports USDA uses to oversee the MGD Program, and met with USDA officials to determine the status of corrective actions to address those internal control weaknesses. We did not verify the accuracy of information included in the financial and progress reports. Additionally, we convened a roundtable of 9 current and former NGO implementing partners of the MGD Program and the Global Food for Education Initiative pilot to obtain their views on the implementation of the program. Finally, we compared USDA's oversight and internal control practices to our Standards for Internal Control in the Federal Government and the American Evaluation Association's An Evaluation Roadmap for a More Effective Government. We conducted this performance audit from May 2010 to May 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. (Appendix I provides a detailed discussion of our objectives, scope, and methodology.) Background: In recent years, international donors, including the World Bank, and national governments in developing countries have increasingly embraced school feeding. According to the World Bank, low-income countries are expanding school feeding partly in response to United Nations Millennium Development Goals addressing hunger and education.[Footnote 4] However, views on the benefits and costs of school feeding vary. We conducted a review of selected international school feeding studies completed over the past 20 years. The studies generally found that school feeding programs increase students' total food consumption and attendance. However, we found mixed results in these studies on the effects of school feeding on enrollment, educational progress, and nutrition. The results suggest that school feeding programs do not always achieve the same effect, and various factors, such as the modality of school feeding (whether the program provides in-school meals, take-home rations, or both), gender of the beneficiaries, and type of food provided, can influence the programs' outcomes. See appendix II for details on our review of studies on school feeding programs. The MGD Program Has Evolved Since the Inception of the Pilot: The MGD Program promotes education, nutrition, and food security for poor children in low-income countries that have low literacy and primary school completion rates. Specifically, the program provides donations of U.S. agricultural commodities,[Footnote 5] as well as financial and technical assistance, for school feeding and maternal and child nutrition projects in low-income countries. Administered by USDA's FAS, the program is implemented by NGOs and WFP which generally support school feeding and complementary activities at numerous schools in targeted communities. The MGD Program was preceded by the pilot known as the Global Food for Education Initiative. The Initiative was proposed by President Clinton in 2000. As indicated in figure 1, under the pilot, USDA donated $300 million in U.S. surplus commodities through the Commodity Credit Corporation (CCC) to implement the school feeding program for 1 year. [Footnote 6] In practice, implementation in some cases took longer, and the pilot ended in 2003. The Initiative's objectives were to improve student enrollment, attendance, and performance in poor countries. Congress authorized the MGD Program in 2002, expanding its objectives to include improved maternal and child nutrition and the achievement of food security, and reauthorized it in 2008 through fiscal year 2012. The program has been subject to annual appropriations after fiscal year 2003, and between fiscal years 2003 and 2010 the annual program funding ranged from $50 million to $210 million. The total amount of funding provided for the Global Food for Education Initiative and the MGD Program from 2000 through 2010 is about $1.2 billion. Appropriated funds for the MGD Program are no-year appropriations; therefore, the funds are available for obligations without fiscal year limitations. In fiscal year 2010, funding for the MGD Program constituted less than 10 percent of the U.S. government's overall funding for international food aid. Figure 1: Evolution of the MGD Program: [Refer to PDF for image: timeline] Pilot: Major Events: 2000: Global Food for Education Initiative (pilot) initiated; Funded by $300 million in surplus commodities through 2002; Funded by Commodity Credit Corporation (CCC) surplus commodity. Major Objectives: To carry out an international school feeding program to improve student enrollment, attendance, and performance in poor countries. Major Events: 2001-2002: The MGD Program authorized by the Farm Security and Rural Investment Act of 2002 with $100 million in CCC funds for 2003; Major Objectives: Legislation states to carry out preschool and school food for: * Education programs in foreign countries to improve literacy and primary education, particularly with respect to girls; * Maternal, infant and child nutrition programs for pregnant women, nursing mothers, infants, and children who are 5 years of age or younger; * Improving food security, and reducing the incidence of hunger. Appropriations/funding: Major Events: 2003: Regulations governing the program published; $100 million CCC Funds. Major Events: 2004: $50 million appropriated. Major Events: 2005: $87.5 million appropriated. Major Events: 2006: $100 million appropriated. Major Events: 2007: $100 million appropriated. Major Events: 2008: The MGD Program reauthorized by the Food, Conservation, and Energy Act of 2008; $100 million appropriated. Major Events: 2009: New regulations require third party interim and final evaluations; $100 million appropriated; $84 million CCC Funds; Major Objectives: Agency guidance outlines six operational objectives: 1. increased enrollment and attendance rates, especially for girls; 2. improved student achievement levels through improvements in the learning environment; 3. improved maternal, child and student health and nutrition; 4. attracting non-FAS contributions to development activities; 5. enabling community support for infrastructure development; 6. increased government and community support in education. Major Events: 2010: The appropriation includes $10 million for a micronutrient pilot; $209.5 million appropriated. Major Events: 2011: USDA’s proposal solicitation requires implementing partners to focus on three broad objectives; $199.5 million appropriated; Major Objectives: The proposal solicitation has three broad objectives: 1. improved educational quality; 2. improved nutrition; 3. sustainability. Source: GAO analysis of legislation and USDA data. [End of figure] The MGD Program has experienced some changes in recent years. Initially, USDA signed single-year agreements with implementing partners. In 2006, it began offering three-year agreements, and most agreements signed in 2010 were for three years. USDA's guidance includes six operational objectives, as indicated in figure 1, which in fiscal year 2011 were subsumed into three broad objectives of education, nutrition, and sustainability of school feeding. USDA now refers to the six operational objectives as intermediate results. [Footnote 7] The MGD Program authorizing statute encourages the program to focus on girls since they tend to have low school attendance rates, and education[Footnote 8] of girls benefits the entire family. The program also offers nutrition[Footnote 9] assistance to undernourished mothers and their preschool-age children to improve the health and learning capacity of the children before they enter school.[Footnote 10] In addition to food, the MGD Program provides financial assistance to fund complementary school feeding activities, such as teacher training, nutrition education, and fostering parental involvement. The legislation stresses sustainability and requires that all agreements specify a timeline to achieve graduation.[Footnote 11] Management and Funding of the MGD Program: At FAS, the MGD Program is administered by the Food Assistance Division (FAD), whose staff in Washington, D.C., manage the program and conduct monitoring site visits. Two other units at FAS have some oversight responsibilities over USDA's international food assistance programs, including the MGD Program--Monitoring and Evaluation Staff (MES) and the Compliance, Security and Emergency Planning Division (Compliance). MES was established in 2006 to conduct monitoring and evaluation of USDA's capacity building and development programs and to close out food assistance agreements. With respect to the MGD Program, MES has primarily focused on closeout of NGO-implemented agreements. Compliance conducts occasional risk-based financial compliance reviews of NGO-implemented agreements; these reviews may include visits to NGO headquarters or field offices. In addition, FAS's agricultural attachés at overseas posts may provide comments on MGD Program proposals. The MGD Program is implemented by NGOs and WFP. NGOs and WFP submit proposals based on criteria established by USDA. See appendix III for a discussion of how USDA determines the priority countries and establishes criteria for proposal solicitations. USDA requires NGOs implementing MGD projects to submit semiannual and annual reports, such as project status and audit reports, whereas WFP prepares an annual standard project report.[Footnote 12] USDA staff at FAD, MES and Compliance review these reports to track program performance and compliance with financial rules and regulations. See appendix III for a description of the reports implementing partners are required to submit to USDA. Since 2003, MGD Program funding has supported school feeding and related activities in 41 countries, as seen in figure 2. In 2010, the program was implemented in 28 countries. From 2003 through 2010, $475 million (or over 50 percent) of MGD Program funding was directed to Africa, $252 million to Asia, $117 million to Latin America and the Caribbean, $25 million to the Middle East, and $23 million to Eastern Europe. See appendix IV for MGD Program funding by year, country, and region. Figure 2: MGD Program Countries Since 2003: [Refer to PDF for image: illustrated world map] 41 Countries Have Had School Feeding Programs: 28 Countries with MGD projects in fiscal year 2010: Afghanistan; Bangladesh; Benin; Bolivia; Cambodia; Cameroon; Chad; Ethiopia; Gambia; Guinea-Bissau; Guinea; Guatemala; Honduras; Kenya; Kyrgyz Republic; Laos; Liberia; Pakistan; Senegal; Sierra Leone; Cameroon; Angola; Mali; Malawi; Madagascar; Mozambique; Niger; Republic of Congo. 13 Countries that no longer had MGD projects in fiscal year 2010: Albania; Bhutan; Dominican Republic; Eritrea; Ghana; Ivory Coast; Lebanon; Moldova; Nepal; Nicaragua; Tanzania; Rwanda; Vietnam. Sources: USDA (data); Map Resources (map). [End of figure] Based on USDA's data, since the MGD Program was initially authorized in 2002, WFP received 43 percent of the total funding and NGOs 57 percent. Twenty-two NGOs have implemented the MGD Program since 2003. In 2010, WFP invested about $500 million in its school feeding program, with the MGD Program contributing about 15 percent of the funding. USDA Has Established a Grant Approval Process Consistent with the MGD Program's Objectives, but Oversight Weaknesses Make It Difficult to Determine the Extent to which the Program Is Achieving Its Objectives: USDA has established a grant approval process consistent with achieving the MGD Program's objectives of education, nutrition, and sustainability, but weaknesses in USDA's oversight of the program limit its ability to determine the extent to which those objectives are actually met. USDA has developed criteria for designating priority countries, assessing proposals, and negotiating grant agreements, and reports that the MGD Program has fed an annual average of almost 3 million beneficiaries from fiscal year 2003 to 2010 by providing in- school feeding and take-home rations. In addition, USDA supports activities that are complementary to school feeding, such as teacher training, health and nutrition education, and fostering parental and community involvement. However, we identified weaknesses in USDA's performance monitoring and evaluation. For example, USDA conducts limited monitoring in the field resulting in a reliance on the implementing partners' performance monitoring, whose rigor varies by implementing partner. In addition, USDA has not prioritized evaluation of completed MGD projects to assess the program's strengths and weaknesses. The MGD Program Includes Three Phases: Grant Approval, Monitoring, and Evaluation: Our analysis of USDA's oversight of the MGD Program examines three broad phases: grant approval, monitoring, and evaluation. Figure 3 describes the three phases of the program's oversight. Appendix III provides a description of the MGD Program's operation, including the grant approval process. Figure 3: Key Elements of the Grant Approval, Monitoring, and Evaluation Phases: [Refer to PDF for image: illustration] Phase 1: Grant Approval: * Designate program objectives and priority countries; * Solicit proposals to address program objectives; * Review proposals in light of program objectives and other relevant criteria; * Negotiate agreements based on proposals that were highly rated in the review process. Phase 2: Monitoring; * Ongoing and systematic collection and verification of data to determine whether programs are being implemented as intended; * Track and report progress using preselected indicators throughout the life of the program. Phase 3: Evaluation; * Develop policies and procedures to guide evaluations; * Develop evaluation plans and set priorities; * Assess the strengths and weaknesses of programs to improve their effectiveness; * Use systematic data collection and analysis to address questions about how well programs and policies are working, whether they are achieving their objectives, and why they are or are not effective. Sources: GAO analysis of GAO, USDA, WFP, Committee of Sponsoring Organizations of the Treadway Commission, and American Evaluation Association data. Note: The standards or sources used include the following: * USDA's criteria for designation of priority countries and proposal assessments. * USDA's regulations governing the McGovern Dole International Food for Education and Child Nutrition Program (7 CFR 1599). * GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). * Committee of Sponsoring Organizations of the Treadway Commission, Internal Control - Integrated Framework, Sept. 1992. * GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). * American Evaluation Association, An Evaluation Roadmap for a More Effective Government, 2010: * GAO, Program Evaluation: Experienced Agencies Follow a Similar Model for Prioritizing Research, GAO-11-176 (Washington, D.C.: Jan. 14, 2011). * GAO, International Food Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts, GAO-09-980 (Washington, D.C.: Sept. 28, 2009). * GAO, Performance Measurement and Evaluation: Definitions and Relationships, GAO-05-739SP (Washington, D.C.: May, 2005). * United Nations World Food Program, Office of Evaluation, Monitoring and Evaluation Guidelines. Based on the standards and sources cited, GAO determined that the points reflected on this graph represent key elements of the grant approval, monitoring, and evaluation phases of the MGD Program. [End of figure] USDA Has Established a Grant Approval Process Consistent with MGD Program Objectives: USDA established a grant approval process to help ensure that projects likely to achieve the MGD Program's objectives of education, nutrition, and sustainability are approved. As such, USDA has developed criteria for designating priority countries, soliciting proposals, assessing proposals, and negotiating grant agreements, as shown in figure 4. Figure 4: The Grant Approval Phase: [Refer to PDF for image: illustration] Phase 1: Grant Approval-Key Elements: * Designate program objectives and priority countries; * Solicit proposals to address program objectives; * Review proposals in light of program objectives and other relevant criteria; * Negotiate agreements based on proposals that were highly rated in the review process. Grant Approval of the MGD Program: * USDA's current MGD objectives are nutrition, education, and sustainability; * Every year, USDA designates priority countries and issues a solicitation for proposals that specifies the criteria for program selection; * USDA has developed a process to assess the program proposals that are submitted; * The MGD Program agreements generally include activities to address all program objectives. Source: GAO analysis of USDA data. Note: Based on USDA's regulations governing the MGD Program (7 CFR 1599), USDA's criteria for designation of priority countries, proposal solicitation, standards for proposal evaluation, and grant negotiation, GAO determined that the points reflected on this graph are key elements of the grant approval phase for the MGD Program. [End of figure] MGD Program Grant Approval Process Generally Supports the Objectives of the Program: Annually, according to FAS, it selects priority countries that would most benefit from the MGD Program based on criteria that are intended to help ensure that the priority countries are needy and are committed to improving their education systems. To help promote sustainability, countries with existing projects are also considered priority countries. The legislation governing the MGD Program states that an agreement with an eligible organization should include provisions to sustain the benefits of the program and estimate time frames for achieving sustainability, among other things. See appendix III for criteria used to determine priority countries. Based on USDA's proposal solicitation criteria for fiscal years 2008 through 2010, the MGD proposals should identify developmental goals for improving literacy and primary education (especially for girls). For example, USDA criteria stated that the organization should include data on the current primary school attendance and completion rates for the target population, and explain how the proposed project would help to increase these rates. For maternal and child nutrition activities, the applicants' proposals are also expected to demonstrate how the project will improve the food security and nutritional status of the target population. Further, USDA required the NGOs and WFP to delineate a plan for achieving sustainability of school feeding through the involvement of the government, local institutions, and communities.[Footnote 13] For example, the applicant's proposals are expected to encourage partnerships via the donation of counterpart funding, in-kind materials, and labor and space to sustain project activities. Fifteen percent of each proposal's evaluation score is based on whether the applicant is able to attract country, community, or other donor contributions to the project supported by the MGD Program. See appendix III for criteria used to assess the grant proposals. In assessing the proposals, USDA's program analysts consider various factors, such as projects' outcomes, need, and implementation capacity. We determined that USDA's program analysts used a standard form designed to ensure that the need for the proposed projects was justified, the key factors for consideration were rated on a common scale, and the strengths and weaknesses of the proposals were discussed. From our review of USDA's assessment, we determined that the MGD Program's objectives of education, nutrition, and sustainability were considered in the assessment, though the education and nutrition goals were addressed more generally through discussions of program need and overall proposal quality, while sustainability was an element that received a specific rating. Our analysis of USDA's assessments of the 2010 proposals found that most of the analysts made written comments on the MGD Program's sustainability. Seven of the 18 programs received positive comments about their potential sustainability while 4 received negative comments. An additional 4 programs received mixed comments about their prospects for sustainability. Our review of these proposals found that the analysts had generally completed the assessments as required and provided justifications for funding the projects in question. Implementing Partners Provide Specific Details on In-School Feeding and Take-Home Rations: The next stage of the grant approval process involves negotiating agreements between USDA and the NGOs. To obtain an understanding of this stage and to determine whether or not the agreements contained elements consistent with the MGD Program's stated goals, we reviewed all 31 agreements that USDA entered into with NGOs and WFP in fiscal years 2008 through 2010.[Footnote 14] Key areas covered in the agreements include the food that will be provided to the beneficiaries, the methods used to select the beneficiaries, the activities that will be undertaken to complement the school feeding program, the types of MGD Program performance indicators for measuring progress, and coordination with governmental or nongovernmental entities. USDA allows implementing partners the flexibility to adapt their projects to the local context in terms of establishing the type of school meals provided, determining appropriate ration sizes, providing take-home rations, and choosing which commodities to provide for the meals. Our review of NGO and WFP agreements indicate that all but 3 of the 31 projects planned to provide in-school feeding.[Footnote 15] The implementing partners did not always specify the in-school meals they would provide, but when they did, they reported providing breakfasts, lunches, or snacks. In addition, 19 of the 31 agreements planned to provide take-home rations, most typically to girls, but also to teachers and other caregivers in some projects. Further, about half of the 19 NGO agreements contained a provision for school feeding for preschoolers, and one of the WFP agreements contained such a provision.[Footnote 16] The Process of Selecting Beneficiaries Varies Between NGOs and WFP: Based on our review of the NGOs and WFP's agreements, we found that all projects considered nutritional and health needs when selecting beneficiaries. NGOs agreed to use a wide range of criteria to select beneficiaries while WFP agreed to use a more standardized process. USDA does not provide guidance on selecting beneficiaries, allowing implementing partners to use a diverse set of criteria to select beneficiaries. The NGOs typically rely on the host government and the opinions of local stakeholders, as well as a variety of statistics to help identify communities and schools for their projects. Examples of these statistics are low levels of enrollment and low levels of food security. Some of the MGD program agreements for the NGOs planned to select schools that required new facilities, teacher training, or the development of parent teacher associations or school management committees, while others planned to select schools that already had existing facilities or community organizations. In some instances, the schools that had the existing facilities were ones that already had projects, and for which continuations were being requested. See appendix V for details on the methods NGOs agreed to use to select beneficiaries. According to WFP, its school feeding programs target food insecure areas, which tend to be rural, identified through food security assessments. Food security information is combined with the government-provided education-related information (such as enrollment, attendance, retention, and completion rates). WFP includes educational indicators into its food security assessments to inform its decisions on school feeding and is now trying to standardize this process by using a set of targeting guidelines specific for school feeding. Implementing Partners Agreed to Undertake Various Complementary Activities to Achieve the MGD Program's Objectives: The implementing partners agreed to undertake various complementary nonfeeding activities to achieve the MGD Program's objectives. For example, based on our analysis of all 31 NGO and WFP agreements from fiscal years 2008 through 2010, we found that MGD projects planned to support an average of 9 complementary activities. Figure 5 provides examples of the top three activities that NGOs and WFP agreed to undertake for each objective. In addition, appendix VI provides more details about the complementary activities that addressed these objectives. Figure 5: Examples of Complementary Activities in Agreements with Implementing Partners: [Refer to PDF for image: illustration of three objectives with photos for each objective from Cambodia, Guatemala, and Kenya] Objective: Education: Number of agreements, proposals, or country programs with at least one activity for this objective: 26. Examples of complementary activities: Teacher training[A]: 14; General school facilities: 13; Educational supplies/equipment: 12. Objective: Nutrition: Number of agreements, proposals, or country programs with at least one activity for this objective: 29. Examples of complementary activities: Health/nutrition education: 22; School facilities: potable water: 16; School facilities: latrines: 14. Objective: Sustainability: Number of agreements, proposals, or country programs with at least one activity for this objective: 29. Examples of complementary activities: Sustainability at the local level: 17; Parent and community involvement in the schools: 17; Sustainability at host government level: 11. Average number of activities per agreement/proposals, or country programs: 9. Sources: GAO analysis of NGOs’ agreements and WFP’s proposals and country programs from fiscal years 2008 through 2010. Note: These examples are based on our analysis of19 NGO agreements and 12 WFP agreements, proposals, and country programs. We categorized activities into education, nutrition, and sustainability based on our analysis of each activity's relevance to each objective. We did not categorize activities that either addressed multiple objectives or addressed other objectives. [A] The numbers in the figure represent the number of NGO agreements and WFP agreements, proposals, and country programs that planned to undertake those activities. [End of figure] In order to help countries or projects achieve sustainability, the agreements listed other organizations that planned to make contributions. Some organizations provided a dollar value for the planned contributions from sources other than MGD Program funding. Among the 19 NGO agreements we analyzed, 5 reported no dollar amounts for additional contributions from other sources; 9 indicated that the contributions would be between $10,000 and $600,000; and 5 reported that the contributions would be more than $5 million. These contributions represented between less than 1 percent and more than 74 percent of the funds provided by USDA for the projects. Due to differences between WFP and NGO projects, and the way that the WFP program reported other donor contributions, we were not able to make direct comparisons between WFP and NGOs in this regard.[Footnote 17] USDA Has Developed a Set of Performance Indicators to Track MGD Program Performance by Program Goals: Based on our review of the grant agreements, USDA currently has more than 30 performance indicators for the MGD Program. USDA's performance indicators include both output and outcome measures. Output measures track the direct products and services delivered by a program while outcome measures track the results of the products and services delivered.[Footnote 18] Our analysis of all 31 NGO and WFP agreements found that the implementing partners agreed to use an average of 22 of these indicators. Table 1 shows the MGD Program's performance indicators, and appendix VII lists these indicators along with the number of NGO agreements and WFP proposals that include each indicator. Table 1: MGD Program Performance Indicators in NGOs Agreements and WFP Proposals: Performance Indicators: * Educational access; * Percentage increase in enrollment--boys; * Percentage increase in enrollment--girls; * Percentage increase in attendance--boys; * Percentage increase in attendance--girls; * Percentage increase in attendance (gender not specified); * Number of take-home rations distributed; * Number of food supplements provided. Educational progress: * Percentage of boys and girls entering grade 1 of primary school who reach grade 3 (or grade 5) multiyear agreements (cohort survival to grades 3 or 5); * Promotion rate; * Continuation rate; * Percentage of targeted schools with adequate school supplies; * Percentage of targeted schools with printed materials to support literacy and numeric instruction; * Percentage/number of teachers receiving training; * Number of communities with adult literacy classes; * Number of supply kits provided. Nutrition: * Number of daily meals provided; * Number of take-home rations distributed; * Percentage of participating schools implementing health and nutrition education for students; * Percentage of participating schools with established prophylactic programs; * Percentage of participating schools with established health care provider to the schools including preschools; * Percentage of mother and child clinics/health facilities supported; * Percentage of children on target with age-to-weight and height growth; * Percentage of schools that institutionalize health and hygiene programs. Sustainability: * Number of other donors contributing to ancillary projects of complementary activities (school infrastructure, immunization program); * Dollar amount of donor contributions; * Percentage of cost sharing; * Percentage of donor support; * Implementing partner's/participant's/organization's independent contribution; * Percentage of schools where parents provide a defined level of support; * Percentage of schools with parent groups (associations or councils) that play a defined role in program management; * Number of complementary programs; * Number of potable water projects; * Number of latrine and/or kitchen repairs and constructions; * Number of nutrition and health education classes for adults; * Percentage of schools transitioning into parent-teacher association/ government-supported feeding program; * Percentage of increased government support to education sector; * Number of training sessions for capacity building; * Percentage of donor support vs. indigenous support; * Number of provincial Ministries of Education that received training in subjects, such as community mobilization, administration, and monitoring. Source: GAO analysis of NGOs agreements and WFP proposals from 2008 through 2010. [End of table] Implementing Partners Planned to Work with Host Governments but Reported Limited Coordination with U.S. Government Agencies: As part of the grant approval process, the NGOs and WFP also provide information regarding activities that other entities are undertaking in the recipient countries to address poverty, hunger, and deficient primary education, and whether the proposed project complements or duplicates those activities. Based on our analysis of the agreements covering fiscal years 2008 through 2010, all the NGOs and WFP reported they would work with host government agencies. In particular, 27 of the 31 NGOs and WFP agreements indicated they would work with the host government's Ministry or Department of Education, and 17 of 31 indicated they would work with the host government's Ministry of Health to further the projects' goals. A few NGOs agreements indicated they would work with United Nations organizations, specifically WFP (2 of 19) and UNICEF (2 of 19). All 12 WFP agreements indicated they would work with UNICEF. Five (5) of the 31 NGOs and WFP agreements, proposals, and country programs indicated that they would coordinate with USAID in some way, for example, through an in-country coordinating committee or by working with another partner that was funded by USAID.[Footnote 19] USAID administers multiple programs whose goals are similar to those of USDA's MGD Program and aim to improve food security, maternal and child health, education (including food for education), and nutrition in developing countries.[Footnote 20] However, USDA's coordination with USAID on these programs is limited. USDA obtains input from some offices at USAID when it assesses proposals for the MGD Program, but it has not systematically considered how it could more effectively leverage USAID's extensive overseas presence, or benefit from USAID's expertise in basic education. An official at USAID's Office of Education informed us that she has never been requested or invited to review MGD Program grant proposals or provide input to performance indicators. USAID's Office of Education has developed a set of learning assessment tools to measure educational progress, but USDA has not determined whether these tools can be useful in monitoring educational progress for the MGD Program. However, a USDA official said that FAS plans to explore the usefulness of these assessment tools for the MGD Program. USAID's budget for basic education in fiscal year 2010 was about $943 million. In fiscal year 2010, USAID's basic education program was implemented in 17 of the 28 countries where the MGD Program was implemented. In addition, some officials we interviewed during our fieldwork stated that more coordination was needed among the donors at the country level. For example, USAID and USDA officials noted that even when both agencies had similar programs in the same country, their coordination was limited. Moving forward, USDA expects greater coordination among host government and U.S. government agencies on the MGD Program. In fiscal year 2011, USDA, for the first time, reflects coordination as a separate category in its solicitation for proposals and has assigned a value of 10 percent to this category.[Footnote 21] USDA's Performance Monitoring of the MGD Program's Implementation Is Weak: Our analysis of the monitoring phase includes the review of key elements of monitoring, such as ongoing and systematic collection and verification of data, as well as the tracking and reporting of progress using pre-selected indicators, as shown in figure 6. Figure 6: The Monitoring Phase: [Refer to PDF for image: illustration] Phase 2: Monitoring-Key Elements: * Ongoing and systematic collection and verification of data to determine whether programs are being implemented as intended. * Track and report progress using preselected indicators throughout the life of the program. Monitoring of Implementation: * Reviews implementing partners’ reporting: – USDA does not systematically analyze implementing partners' reports; – USDA provides limited feedback to implementing partners; – Reporting requirements do not ensure implementing partners collect and submit consistent and complete information; – USDA’s plans for improvement: implementing a new information system to reduce inconsistencies in reporting, providing training and assistance to NGO staff, requiring the implementing partners to report on the same set of indicators, and introducing the Results-Oriented Management process. * Performance monitoring in the field: – USDA's overseas staff provide limited monitoring support in the field; – USDA’s headquarters staff do not conduct systematic site visits in the field; – USDA relies on implementing partners’ performance monitoring in the field; – USDA has recently increased its headquarters staff and conducted more sites visits in the field. Sources: GAO analysis of GAO, USDA, WFP, Committee of Sponsoring Organizations of the Treadway Commission, and American Evaluation Association data. Note: Based on the standards and sources cited, GAO determined that the points reflected on this graph represent key elements of the monitoring phase of the MGD Program. [End of figure] USDA Does Not Systematically Analyze Implementing Partners' Reporting and Provides Limited Feedback on the MGD Program's Progress: USDA does not systematically analyze implementing partners' reporting to help ensure they are taking the actions agreed upon to achieve the MGD Program's objectives of education, nutrition, and sustainability. In addition, USDA provides limited feedback to implementing partners on the progress reports they submit periodically to USDA. Standards for Internal Control in the Federal Government[Footnote 22] states that activities need to be established to monitor performance measures and indicators and that these controls could call for comparisons and assessments relating different sets of data to one another so that analyses of the relationships can be made and appropriate actions taken. Beyond tallying MGD Program activities such as the number of beneficiaries served or the number of school gardens constructed, USDA has not further analyzed the information reported by implementing partners. For example, USDA currently does not consolidate and analyze the information from separate reports, making it difficult for USDA to track the MGD Program's progress at different points over the life of the agreement or to make comparisons across agreements. In addition, based on USDA's guidance to its staff on reviewing NGOs' reporting, the staff's review generally focuses on the financial and logistical aspects, and to a lesser extent on tracking whether the MGD Program is meeting its objectives. Further, some NGOs reported that USDA provided limited communication and feedback on their progress reporting. For example, one NGO submitted 50-page reports to USDA but received no feedback on its performance indicators. This lack of feedback was also reported by other NGOs. We analyzed the implementing partners' progress reports in our three fieldwork countries of Cambodia, Guatemala, and Kenya since 2004 [Footnote 23] and found that the implementing partners report performance indicators that suggest some progress in achieving the MGD Program's objectives of education[Footnote 24] and sustainability but less progress on the MGD Program's objective of nutrition. This reporting is generated by the implementing partners and the reports accuracy were not verified by USDA. We also did not verify the accuracy of the data contained in these progress reports. Table 2 shows information on the indicators reported by the implementing partners. Table 2: Analyses of Progress Reports Submitted by Implementing Partners in Our Fieldwork Countries: Objective: Educational access[A]; Total number of indicators reported: 74; Number (percent) of indicators with targets: 31 (42%); Number (percent) of targets met: 19 (61%); Number (percent) of indicators that report a baseline value: 67 (91%); Number (percent) of indicators with baseline values that report increases since baseline: 63 (94%). Objective: Educational progress[A]; Total number of indicators reported: 42; Number (percent) of indicators with targets: 34 (81%); Number (percent) of targets met: 21 (62%); Number (percent) of indicators that report a baseline value: 26 (62%); Number (percent) of indicators with baseline values that report increases since baseline: 17 (65%). Objective: Nutrition; Total number of indicators reported: 55; Number (percent) of indicators with targets: 39 (71%); Number (percent) of targets met: 16 (41%); Number (percent) of indicators that report a baseline value: 18 (32%); Number (percent) of indicators with baseline values that report increases since baseline: 10 (56%). Objective: Sustainability; Total number of indicators reported: 87; Number (percent) of indicators with targets: 76 (87%); Number (percent) of targets met: 46 (61%); Number (percent) of indicators that report a baseline value: 44 (51%); Number (percent) of indicators with baseline values that report increases since baseline: 30 (68%). Objective: Total; Total number of indicators reported: 258; Number (percent) of indicators with targets: 180 (70%); Number (percent) of targets met: 102 (57%); Number (percent) of indicators that report a baseline value: 155 (60%); Number (percent) of indicators with baseline values that report increases since baseline: 120 (77%). Sources: GAO analysis of NGOs' logistics and monetization reports and WFP's standard project reports. Note: Our fieldwork countries are Cambodia, Guatemala, and Kenya. [A] USDA's performance indicators for educational access include attendance and enrollment, and its performance indicators for educational progress include promotion and continuation rates, as well as teacher training and the number of schools with adequate supplies. [End of table] USDA's Reporting Requirements Do Not Ensure Implementing Partners Collect and Submit Consistent and Complete Information: USDA's reporting requirements also contribute to the inconsistent information submitted by implementing partners to assess the MGD Program's impact. According to internal control standards, program managers need operational data to determine whether they are meeting their agencies' strategic and annual performance plans and meeting their goals for accountability for effective and efficient use of resources. For example, our analysis of the NGOs' progress reports in our fieldwork countries suggests that the number of performance indicators for educational access and progress that NGOs report ranges from 0 to 17 by each NGO. We also found that one NGO had 8 indicators for nutrition with no targets, and another NGO had 6 indicators for nutrition, all of which had targets. USDA officials agreed that the quality of the reporting varies by NGO, and they are planning to reduce the variation by implementing a new information system (the Food Assistance Information System), providing training and assistance to NGO staff, and revising the reporting forms required of the NGOs. WFP's reporting differs significantly from the NGOs' reporting in our fieldwork countries. As a multilateral organization, WFP currently submits one annual project report to all of its donors.[Footnote 25] The information contained in WFP's annual report differs significantly from the reports submitted by NGOs. First, WFP does not report separately on the MGD Program but reports in the aggregate on its country programs. Its country programs may include other programs and funding by other donors. Therefore, it is often not feasible to isolate the MGD Program's contribution to the reported progress. Second, WFP reported fewer indicators on educational progress and nutrition than the NGOs. In the eight NGOs' progress reports we analyzed, the NGOs reported a total of 40 indicators on educational progress and a total of 39 indicators on nutrition. In comparison, in the six WFP progress reports we analyzed, WFP reported 16 indicators [Footnote 26] on nutrition and only 2 indicators on educational progress, neither of which were reported by NGOs. A senior USDA official stated that starting in fiscal year 2012, USDA plans to require that WFP report on the same set of indicators as the NGOs. USDA does not provide guidance to implementing partners on how to set targets, allowing implementing partners to set their own targets. While the ability for implementing partners to set their own targets may allow them to tailor targets to their specific context, it also hinders USDA's ability to compare performance across NGOs, as NGOs may use varying methods to set targets. USDA plans to address this lack of guidance by developing an indicator handbook that is expected to place a greater emphasis on meeting targets. USDA's reporting requirements also do not ensure that implementing partners collect and submit complete information to assess the MGD Program's impact in education and nutrition. Based on our prior work, [Footnote 27] the ability of a performance indicator to align with program objectives is one key attribute of a successful performance indicator. Our analysis of all 31 NGOs and WFP agreements[Footnote 28] entered into with USDA from fiscal years 2008 through 2010 found that USDA required outcome indicators that measure aspects of educational progress such as promotion and continuation rates, but did not require outcome indicators that measure learning. For example, the implementing partners' progress reports in our fieldwork countries include outcome indicators, such as the number of teachers and principals with a high number of students passing grade 9, but they do not include outcome indicators that directly measure learning, such as achievement tests. For nutrition, our analysis of NGOs and WFP agreements show that only 5 of the 19 NGOs' agreements and none of the WFP proposals and country programs planned to use an outcome measure that directly measures nutrition. For example, except for one indicator (of a total of 55 indicators) in a progress report for a sub- recipient of an implementing partner, the implementing partners' progress reports include indicators such as the number of meals served but did not include any outcome indicators for nutrition, such as weight for age and height for age. Without outcome indicators that directly link to learning and nutrition, it is difficult for USDA to know whether these objectives are being met. USDA seeks to establish a Results-Oriented Management (ROM) process to develop a more comprehensive approach to align the program with its objectives. As part of the ROM, USDA expects to develop a new set of performance indicators for the MGD Program, which would include a set of outcome and output indicators and set both intermediate goals and goals more directly linked to the program's objectives. According to USDA officials, the ROM process is not expected to be implemented before fiscal year 2012. To monitor the MGD Program's progress, USDA requires several reports of NGOs, including the logistics and monetization report and the project status report. In our analysis of these reports in our fieldwork countries, we found that the project status reports contained less quantitative information and were generally duplicative of the logistics and monetization report. See table 7 in appendix III for a detailed description of USDA's various reporting requirements for implementing partners. Project status reports generally contain narratives on educational progress, nutrition, and sustainability. In contrast, the logistics and monetization reports generally contain quantitative information on the MGD Program's progress. Of a total of 78 performance indicators in the project status reports, only 6 targets (8 percent) were reported. In contrast, of a total of 182 performance indicators in the logistics and monetization reports, 138 targets (76 percent) were reported. USDA's Limited Performance Monitoring of the MGD Program in the Field Hinders Its Ability to Verify Implementing Partners' Reporting: USDA has limited field presence to monitor the MGD Program. Specifically, USDA's agricultural attachés[Footnote 29]--responsible primarily for trade promotion and foreign market development in addition to supporting the MGD Program--provide limited support to USDA's monitoring of the program in the field. First, the attachés are stationed in only 7 of the 28 countries where the MGD Program is being implemented.[Footnote 30] In one country, the Republic of Congo, USDA does not have any attaché coverage, even though MGD projects were implemented in the Republic of Congo in 2004 and 2006 through 2011. Other MGD countries without attachés are covered by attachés in other countries in the same region. For example, the attaché in Kenya provides monitoring support to projects in Malawi, Rwanda, Tanzania, and Uganda in addition to Kenya. The agricultural attaché who covers one of our fieldwork countries and is based in another country has only recently begun to travel to our fieldwork country monthly. While in our fieldwork country, the majority of the attaché's time was spent on food safety and trade issues. Second, the agricultural attachés in our three fieldwork countries have generally received no training on how to monitor the MGD Program and conduct limited site visits. In addition, since fiscal year 2009, FAS selects the countries to implement the program based in part on overseas posts' capacity to monitor program implementation. FAS staff at headquarters do not conduct systematic site visits in the field. FAD staff have visited some programs in recent years, but they have not had a systematic approach to monitoring through country site visits that relies on a risk assessment. Internal control standards define risk assessment as the identification and analysis of relevant risks associated with achieving the objectives. Among other things, a site visit based on risk assessment would account for the amount of MGD Program funding to a country, U.S. policy priorities, and whether the implementing partners are new at implementing the MGD Program. Figure 7 shows the number of countries FAD staff visited from fiscal years 2004 through 2010. From 2004 through 2008, the number of countries visited was small relative to the number of the MGD Program countries. For 2009 and 2010, figure 7 shows that FAD increased the number of visits to nine visits per year. According to a senior USDA official, within each country, the MGD Program is generally implemented in hundreds of schools; however, USDA's visits generally lasted 1 to 2 weeks and included visits to a limited number of schools. For example, for Bangladesh, WFP received about $8 million per year in MGD Program funding in 2006, 2008, 2009, and 2010, and planned to serve 350,000 beneficiaries per year from 2008 through 2010, making it a top-five country in terms of the MGD Program's funding. However, FAD did not conduct a site visit to an MGD project in Bangladesh until 2010. Additionally, while on these visits, FAD did not provide guidelines for its staff on how to conduct site visits until about the end of 2010, when it developed draft monitoring guidelines for its headquarters- based staff and attachés. Additionally, FAS's MES officials do not conduct site visits but instead conduct headquarters-based reviews of the documents the implementing partners provide to them. In 2010 and early 2011, FAD hired 6 additional staff, for a total of 14. With these new staff, FAD officials stated they plan to visit every project at least once every 2 years. However, they have not developed a plan for these site visits, and they select and approve countries for visits on a case-by- case basis. As a result of the lack of systematic site visits, by both USDA staff based in Washington, D.C., and in the field, it is difficult for USDA to verify the reporting submitted by implementing partners. Figure 7: Number of MGD Countries Visited and Not Visited, Fiscal Years 2004-2010: [Refer to PDF for image: stacked vertical bar graph] Year: 2004: Number of countries visited by Food Assistance Division staff: 2; Number of countries not visited by Food Assistance Division staff: 20; Total number of MGD countries[A]: 22. Year: 2005: Number of countries visited by Food Assistance Division staff: 1; Number of countries not visited by Food Assistance Division staff: 24; Total number of MGD countries[A]: 25. Year: 2006: Number of countries visited by Food Assistance Division staff: 5; Number of countries not visited by Food Assistance Division staff: 14; Total number of MGD countries[A]: 19. Year: 2007: Number of countries visited by Food Assistance Division staff: 2; Number of countries not visited by Food Assistance Division staff: 22; Total number of MGD countries[A]: 24. Year: 2008: Number of countries visited by Food Assistance Division staff: 1; Number of countries not visited by Food Assistance Division staff: 22; Total number of MGD countries[A]: 23. Year: 2009: Number of countries visited by Food Assistance Division staff: 9; Number of countries not visited by Food Assistance Division staff: 14; Total number of MGD countries[A]: 23. Year: 2010: Number of countries visited by Food Assistance Division staff: 9; Number of countries not visited by Food Assistance Division staff: 16; Total number of MGD countries[A]: 25. Source: GAO based on USDA data. [A] The number of countries for fiscal years 2004 through 2010 are based on USDA's data and include countries with expenditures over $100,000 for the respective year. [End of figure] USDA Relies on the Implementing Partners' Performance Monitoring in the Field: Since USDA has not conducted systematic site visits and performs primarily headquarters-based document reviews of implementing partners' reporting, USDA relies on the implementing partners' performance monitoring in the field. However, USDA has not issued guidance to the implementing partners on their monitoring efforts. This lack of guidance to implementing partners on monitoring has contributed to the varying levels of rigor in plans for monitoring that we found in our analysis of the 19 NGOs agreements from fiscal years 2008 through 2010. In particular, 12 of the 19 agreements did not make reference to monitoring, while only 4 of the 19 agreements provided detailed plans for data collection. A detailed plan would indicate actions such as training staff to collect weight and height for age data or monthly spot checks at schools. With regard to WFP, all 12 projects outlined monitoring plans in their proposals and/or country programs. For example, in Cambodia, the country program reported it would follow a ROM approach, using a toolkit that included standardized reporting forms for all program components. The two programs that did not describe their monitoring activities were different types of WFP programs--protracted relief and recovery operations--which used a somewhat different reporting format.[Footnote 31] The variation in the implementing partners' plans for monitoring contributed to the varying levels of rigor in implementing partners' monitoring we encountered in our fieldwork countries. For example, in one of our fieldwork countries, one NGO reported that it relied on a desk review of progress reports submitted by its local subrecipients and 1-week site visits conducted by the NGO's headquarters-based staff every quarter, at most. This contrasted with the other NGO operating in this country, which reported to have a systematic monitoring system that enabled it to make 2 to 3 spot visits to each school every month. Finally, WFP has its own monitoring system and has issued guidance regarding monitoring of school feeding programs. USDA Has Not Developed Policies and Procedures to Guide Evaluation of Completed Projects: Whereas performance monitoring can address weaknesses that emerge during implementation, evaluating completed projects would provide USDA with essential evidence for improving the MGD Program's effectiveness in education, nutrition, and sustainability, as indicated in figure 8. The American Evaluation Association's (AEA) Evaluation Roadmap for a More Effective Government recommends that each federal agency and its evaluation centers "publish policies and procedures and adopt quality standards to guide evaluations within its purview." Among other things, the policies should include criteria for developing evaluation plans and setting priorities. However, USDA has neither developed policies and procedures to guide evaluation of completed MGD projects nor established priorities for evaluations. [Footnote 32] Figure 8: The Evaluation Phase: [Refer to PDF for image: illustration] Phase 3: Evaluation--Key Elements: * Develop policies and procedures to guide evaluations. * Develop evaluation plans and set priorities. * Assess the strengths and weaknesses of programs to improve their effectiveness. * Use systematic data collection and analysis to address questions about how well programs and policies are working, whether they are achieving their objectives, and why they are or are not effective. Evaluation of Completed MGD Projects: * USDA has not developed policies and procedures to guide evaluation of completed projects. * USDA has not evaluated completed MGD projects. * USDA has not disseminated lessons learned from implementation USDA is taking steps to emphasize evaluation. * In 2009, USDA introduced a requirement for all implementing partners to submit an interim and final evaluation for each MGD project. * USDA is developing plans to conduct its own evaluations of completed MGD projects. Source: GAO analysis of GAO, USDA, and American Evaluation Association data. Note: Based on the standards and sources cited, we determined that the points reflected on this graph represent key elements of the evaluation phase of the MGD Program. [End of figure] USDA has not, until recently, required implementing partners to have their completed MGD projects evaluated. In addition, it has not conducted its own evaluations of these projects. Recognizing the need for evaluation, USDA is taking steps that will emphasize this in the future. First, in March 2009 USDA issued revised MGD Program regulations, which introduced a requirement for all implementing partners to submit evaluations, conducted by an independent third party, at the middle and final points of the implementation period for each MGD Program award.[Footnote 33] According to USDA officials, this requirement applies to all multiyear awards beginning in fiscal year 2010 and USDA expects to receive the first round of final evaluations for these awards in fiscal year 2014.[Footnote 34] USDA's FAD and MES officials are developing guidance for the required evaluations, which USDA plans to release at the International Food Aid and Development Conference in Kansas City in June 2011.[Footnote 35] Our analysis of USDA NGOs agreements and WFP proposals shows various evaluation approaches among MGD Program implementing partners. Most of the NGO agreements we reviewed did not include detailed plans for evaluation. By comparison, most of the WFP proposals and country programs we analyzed discussed evaluation and one WFP proposal included a discussion of a planned joint WFP-World Bank impact assessment [Footnote 36] of its school feeding program in Laos. Second, USDA has not evaluated completed MGD projects, but its MES officials are developing plans to evaluate the MGD Program, as part of the recently initiated ROM process. However, MES has not identified criteria for selecting projects, agency resources available for evaluation activities, or specific time frames to conduct the evaluations.[Footnote 37] AEA's Evaluation Roadmap recommends that in order to develop evaluation plans agencies need to assess what is already known about a program's ability to achieve its objectives. However, MES officials told us they do not analyze evaluations of WFP or NGOs school feeding programs, including assessments of NGO- implemented MGD projects.[Footnote 38] Nor does USDA systematically identify and disseminate lessons learned from the NGOs' experience in implementing the MGD Program. For example, according to NGO representatives who participated in our roundtable, USDA does not facilitate discussion among NGOs about persistent problems, effective practices, or innovative solutions in the MGD Program's implementation. Side bar: Evaluation of School Feeding at the World Food Program: WFP has developed an evaluation policy as well as an evaluation quality assurance system. In addition to evaluations of its emergency and development programs, some of which include school feeding activities, WFP recently initiated a series of impact evaluations of its school feeding activities in Bangladesh, Cambodia, the Gambia, Ivory Coast, and Kenya. The first two studies-—for Cambodia and Kenya, countries where WFP’s school feeding is in part supported by the MGD Program-—were released in 2010. These evaluations employed both quantitative and qualitative methodologies and were conducted by independent consultants. Although these impact evaluations reported that school feeding was associated with higher enrollment, attendance, and completion rates, they also discussed serious constraints in school feeding’s ability to address nutrition, quality of learning, and gender parity in education. [End of side bar] USDA Has Taken Actions to Address the MGD Program's Internal Control Weaknesses, but Improvements Can Be Made: USDA has established internal controls over cash, commodity, and transportation disbursements, including risk-based financial compliance reviews to verify that funds were expended as approved. However, USDA policies for closing out grant agreements do not include time frames for when agreements should be closed. In addition, we found that while the FAS Administrator planned corrective actions to close out USDA's OIG audit recommendations, the Office of the Chief Financial Officer (OCFO) has not received documentation that actions have been taken on 10 recommendations, and 1 recommendation concerning the implementation of a new information system has not been fully implemented. USDA Has Established Internal Controls Over Cash, Commodity, and Transportation Disbursements, and Conducts Risk-based Financial Compliance Site Visits: USDA has established internal controls over cash, commodity, and transportation disbursements for the MGD Program and conducts risk- based financial compliance site visits to verify that funds are spent as approved. We interviewed USDA officials and reviewed USDA's policies and procedures for cash, commodity, and transportation disbursements. We tested the implementation of selected controls including testing three grants that were terminated and determined that expenditures were within budget. We also traced a judgmental sample of key transactions back to the general ledger to verify the accuracy of the program's financial information. However, we did not trace transactions back to source documents. When a MGD Program grant is awarded, USDA negotiates a budget within the grant agreement for cash expenses to be incurred by the NGO for administrative expenses, internal transportation and shipping expenses, and complementary activities such as teacher training, latrine building, and deworming activities. In addition, USDA agrees to supply commodities up to a negotiated tonnage and ocean transport for those commodities. USDA may provide the cash component of the grant as either a cash advance to the NGO or as a reimbursement after the expenses are incurred. In either case, USDA controls seek to ensure funding is provided only for approved expenses. When processing a cash advance or reimbursement, USDA policies direct USDA officials to compare the request for a disbursement against the approved expenses on the grant agreement to ensure that the request is within budget. If the NGO previously received a cash advance or reimbursement, officials also verify that the previous disbursement was properly spent. During this review, if officials discover the NGO has incompletely or improperly spent previously received funds, USDA will request that the NGO submit a reimbursement to USDA. Multiple USDA officials review and sign off before a cash advance or reimbursement is approved and funds are disbursed to the NGO. Once the NGO receives the funds, it is required to obligate all advanced cash within 180 days or return all unused funds within 210 days. USDA has similar processes and procedures in place to ensure that NGOs receive the correct tonnage of commodities and that ocean transport costs are within budget. USDA occasionally conducts financial compliance reviews of its food aid grants to, among other activities, check receipts and verify that the funds were spent as reported to USDA. Site visits are conducted by the Compliance, Security and Emergency Planning Division (Compliance). [Footnote 39] Using a risk-based approach, USDA selects the MGD Program grant to be reviewed from a list of NGOs encountering problems and requiring enhanced monitoring. This list is compiled at periodic meetings held among FAD, MES, and Compliance. NGOs may be on the list for a variety of reasons, including persistent problems with reporting, violations of their agreements, or delayed program implementation. USDA officials stated that MGD Program grants are less likely to have problems than USDA's Food for Progress[Footnote 40] food aid grants for reasons including: (1) the Food for Progress monetization activities are more complex [Footnote 41] and (2) Food for Progress development projects are more varied than MGD projects. Consequently, MGD projects are less likely than Food for Progress projects to be identified as requiring enhanced monitoring or a financial compliance review. The financial compliance reviews generally include a site visit to the NGO's headquarters in the United States, although sometimes, the review is done on-site in the country where the NGO implements the MGD project.[Footnote 42] Before beginning the review, Compliance works closely with the FAD analyst in charge of the grant to determine the NGO's potential compliance problems. As discussed in the previous section, during the grant implementation phase, NGOs are required to submit various reports to USDA that provide financial and programmatic information. Several of the reports have data related to the MGD project expenditures, including: logistics and monetization reports, financial reports, and A-133 audit reports.[Footnote 43] Before carrying out their site reviews, Compliance gives the NGO a 30-to-60- day notice and obtains the logistics and monetization reports and any correspondence and relevant background information the NGO has submitted to FAD. Once on-site at the NGO, Compliance obtains the supporting documentation for expenses funded by the MGD Program grant and verifies that funds were spent as approved by USDA. At the end of the review, Compliance conducts an exit conference with the NGO and reports its findings to FAD. As of January 25, 2011, of 22 closed MGD Program NGO grants that were completed and eligible for financial compliance reviews, Compliance had completed 4 reviews and had initiated 3 more. Two of the financial compliance reviews had no findings. In one, Compliance determined the NGO owed USDA funds because the NGO did not maintain adequate supporting documentation for the program expenses. Another found that the NGO needed to address commodity handling internal control issues. See table 3 for additional information about the 7 financial compliance reviews. Table 3: Results of the MGD Program Financial Compliance Reviews as Reported by USDA: Fiscal year review was conducted: FY2005; MGD Program country: Bolivia; Location where review was conducted: Bolivia; Findings/corrective actions recommended: None. Fiscal year review was conducted: FY2007; MGD Program country: Senegal; Location where review was conducted: Washington, DC; Findings/corrective actions recommended: Insufficient documentation: request for repayment of $2,530.08. Fiscal year review was conducted: FY2010; MGD Program country: Guatemala; Location where review was conducted: Coconut Creek, Florida; Findings/corrective actions recommended: None. Fiscal year review was conducted: FY2010; MGD Program country: Madagascar; Location where review was conducted: Madagascar; Findings/corrective actions recommended: Commodity handling internal control deficiencies: corrective action plan required. Fiscal year review was conducted: FY2010; MGD Program country: Nicaragua; Location where review was conducted: Prattville, Alabama; Findings/corrective actions recommended: Review in progress. Fiscal year review was conducted: FY2010; MGD Program country: Nicaragua; Location where review was conducted: Prattville, Alabama; Findings/corrective actions recommended: Review in progress. Fiscal year review was conducted: FY2011; MGD Program country: Afghanistan; Location where review was conducted: Washington, DC; Findings/corrective actions recommended: Review not yet begun. Source: USDA. [End of table] Although the FAD conducts occasional in-country site visits to assess program activities and Compliance conducts occasional site visits to assess financial compliance, neither verifies that commodities reach the MGD project after arriving at the port of distribution. Currently, once the commodities arrive at port via ocean transport, the NGO or its representative verifies that the correct tonnage is received. If commodities are missing or damaged, the shipping company must reimburse USDA. After the NGO takes possession of the commodities, USDA relies on the NGO to ensure that the commodities reach the intended beneficiaries. FAD officials told us they plan to conduct additional programmatic site visits to evaluate program activities in the future, and guidance for these site visits includes plans to review NGO financial reports to determine whether they accurately reflect expenditures for activities in the field, as well as plans for examining the quality and receipt of commodities. A Compliance official stated that Compliance will continue to use a risk-based approach for selecting NGOs for financial compliance reviews and that as a result, they cannot predict how many MGD Program-related compliance visits will occur per year in the future. USDA Policies on How to Close Out Grants Do Not Include Time Frames for When Grant Agreements Should Be Closed: USDA's draft grant closeout policies do not include time frames for when grant agreements should be closed. As a result, USDA is at risk that grant agreements will not be closed out in a timely fashion. Standards for Internal Control in the Federal Government[Footnote 44] states that managers need to compare actual performance, such as financial performance, to planned or expected results throughout the organization and analyze significant differences. In addition to other activities, grant closeouts compare planned and approved expenditures to actual NGO expenditures and assist USDA to verify that grantees have met financial requirements. USDA grant agreement closeouts are to be completed after the MGD project activities are complete, and the NGO has submitted its final logistics and monetization report. Two divisions in FAS participate in grant closeouts: FAD and MES. FAD staff prepare the closeout package with the documents necessary to close out the grant and MES officials conduct the closeout. MES officials cannot close out a grant until they have received the package from FAD. The closeout process is guided by draft policies and consists of a financial analysis examining reported expenses, as well as a review of the MGD Program's grant agreement objectives against reported outputs and outcomes. Specifically, during the closeout the USDA official reviews NGO- provided reports, such as logistics and monetization reports, financial reports, and A-133 Single Audits, to verify there were no agreement violations and that funds and monetization proceeds were not misused.[Footnote 45] Finally, the official creates a closeout evaluation report. USDA's FAS has been cited by the USDA OIG in the past for having a backlog of food aid grant agreements ready to be closed out, and FAS has responded by prioritizing certain grant closeouts. For example, in response to a 2006 OIG finding, FAS closed the remaining fiscal years 1998-2001 food aid grants. The OIG report also noted that FAS should develop procedures to ensure that agreements for fiscal year 2002 and beyond receive timely closeout reviews. In response, USDA set and achieved a goal to close out 60 percent of all agreements for fiscal years 2002 and 2003. The OIG noted that unless an evaluation or closeout review has been conducted for prior agreements, FAS is unable to confirm that a NGO has been compliant and FAS cannot fairly judge whether the NGO has committed any violations that would preclude a favorable consideration for new grants. USDA has made progress in closing out backlogged grants; however, we found that of the 42 MGD Program grants awarded before 2008, 15 (36 percent) have completed program activities but have not been closed. Two of the 15 grants remain open because USDA terminated the grant agreements and is in the process of investigating how much of the grant the NGOs must reimburse to USDA. For 7 of the 15, USDA has all the necessary documentation, but has not closed out the grant. For the remaining 6 open grants, which were awarded in 2005 and 2006, USDA is still organizing the paperwork received from the NGOs that is necessary to conduct the closeout. According to an OIG report, USDA has set a goal to close 80 percent of grant agreements within 6 years of signing; however, a senior FAD official told us FAD had no official goal or policy governing the time frame within which closeouts should be completed. Closeouts are an important grant management procedure because they are the final point of accountability for grantees.[Footnote 46] Additionally, during the closeout process, USDA officials reported that sometimes they discover that NGOs owe USDA funds. Funds returned from NGOs can be used by USDA to ensure it meets its obligations to other NGOs. For instance, in cases where commodity or transportation costs rise unexpectedly during the course of a grant agreement, USDA can use the reimbursed funds to cover unanticipated cost increases and to help ensure delivery of commodities to MGD projects. As of February 15, 2011, 15 NGOs that were awarded grants between 2003 and 2006, reimbursed USDA over $852,000 in unspent or misused funds. We found one instance in which a NGO retained $23,500 of USDA funds for 53 months, well after its grant activities had been completed, before USDA received the reimbursement. In general, the funds returned to USDA make up a small proportion of the overall funding advanced to the NGO--on average 3.6 percent--however, in some cases, the amount can be significant. For example, two NGOs returned approximately $249,000 to USDA, or over 34 percent of the total USDA funding they had received. NGOs are not required to pay interest on unspent or misused funds that they return to USDA if the NGO returns the funds within 30 days of being notified by USDA that funds are due. Timely closeouts provide valuable information to USDA as it decides whether or not to fund future NGO activities, and USDA officials told us that closeouts provide information that informs their future grant decisions. USDA Has Not Fully Implemented and Completed All USDA OIG Audit Recommendations: USDA has not fully implemented and completed all USDA OIG audit recommendations. The OIG released audits on FAS's food aid programs in 2006 and 2007.[Footnote 47] Sixteen of the OIG audit findings and related recommendations pertained to the MGD Program, and 11 of 16 recommendations remain open, including the implementation of a new Food Aid Information System (FAIS) that will integrate financial management components and has been partially implemented, according to USDA officials.[Footnote 48] The FAS Administrator has developed corrective actions to close out USDA OIG audit recommendations; however, the OCFO has not received and/or reviewed documentation that the corrective actions have been taken and therefore the recommendations are not considered complete. Office of Management and Budget Circular A-50 states that agencies are responsible for ensuring that corrective actions are taken on audit recommendations. In order for FAS to fully implement and officially close an audit recommendation, FAS must present appropriate evidence to the OCFO that planned corrective actions have been taken on the finding. The OCFO then needs to review and approve this documentation before closing the recommendation as fully implemented. According to a FAS official, USDA has taken action on all 16 OIG MGD Program-related recommendations and has completed 5 of them. For example, the OIG recommended that USDA review NGO reports to track problems, such as incomplete reports. Recognizing the need for early detection of problems in grant agreements, USDA agreed with the recommendation, and during fiscal year 2006 a staff member was assigned to review each report and quickly note any obvious problems. Additionally, USDA stated that it would give priority to reviewing reports from NGOs already experiencing problems. Subsequently, the OCFO closed this OIG recommendation. However, of the 11 remaining open recommendations, 10 are still open because the OCFO has not received documentation from the FAS Administrator demonstrating that planned corrective actions have actually been taken. For example, the OIG determined that FAS did not have a system to identify problematic NGO agreements and perform regular reviews of these agreements. FAS agreed with this finding and, in response, set the following criteria to identify problematic agreements: monetization problems; high-risk country; inconsistent reporting; unexplained program delays; and warnings from in-country USDA staff, anonymous whistleblowers, or other sources. However, USDA has not provided the OCFO with documentation to close this recommendation. Additionally, USDA has not provided documentation on corrective actions taken related to new procedures for on-site reviews and revised procedures for establishing timely closeouts of food aid agreement operations. The last open recommendation concerns the implementation of FAIS. This new system is expected to address security control weaknesses and other internal control issues. FAS had originally told the OIG that it hoped to have the system implemented by January 2009, however USDA officials stated that the system implementation was delayed due to budget, contracting, and other delays. FAS has not yet completed implementation of FAIS, but it has taken other interim steps to address financial management weaknesses. For example, according to USDA officials, the current financial management system has internal control weaknesses related to tracking expenditures at the grant agreement level. USDA's accounting functions for the MGD Program are performed by USDA's Farm Service Agency, which previously did not report individual transactions at the grant agreement level, thus preventing USDA from closely monitoring grant financial transactions. As an interim solution, the Farm Service Agency created a series of manual spreadsheets that enable USDA officials to track financial and nonfinancial transactions at the grant agreement level. These spreadsheets compile data including expenditures and remaining amounts authorized for disbursements. FAIS is expected to strengthen internal controls, including automating these manual financial processes. In addition, FAIS will result in improvements in the reconciliation and reporting of unobligated balances. Further, FAIS is expected to result in improved oversight of NGOs and food aid agreements. The new FAIS system will contain information related to the grant process, including: program budgets, project proposals, and agreements. It will also capture information related to grant implementation, including: commodity and freight purchases, commodity shipments, and payments. Additionally, FAIS will capture financial management information from NGO logistics and monetization reports and financial reports. FAS plans to implement FAIS in three phases: March 2011, July 2011, and September 2011. FAS met its deadline to commence implementation in March 2011 and agency officials state the system is scheduled to be fully implemented by September 2011 as planned. Conclusions: Although USDA has established a grant approval process consistent with the MGD Program's goals of education, nutrition, and sustainability, weaknesses in USDA's oversight of the program limit its ability to ensure that those objectives are actually met. Specifically, USDA's inadequate monitoring of implementing partners' performance raises concerns about its capacity to identify critical operational problems and take real-time corrective action. Moreover, the lack of an evaluation system, including policies and procedures, limits USDA's ability to identify the strengths and weaknesses of the MGD Program, take appropriate actions, and identify and disseminate lessons learned. USDA is taking steps to improve its financial oversight of MGD Program grants. However, further improvements would help strengthen internal controls. Appropriate financial internal controls are essential for ensuring taxpayer dollars are spent as intended. USDA has controls in place over MGD Program expenditures, but a lack of timely grant closeouts prevents USDA from ensuring that grantees have met all financial requirements and that unused or misused funds are promptly reimbursed to USDA. Recommendations for Executive Action: To improve USDA's oversight of the MGD Program in the areas of monitoring, evaluation, and financial management, we recommend that the Secretary of Agriculture direct the Administrator of the Foreign Agricultural Service to implement the following three recommendations: * Establish a monitoring process that would systematically analyze and report on a preselected set of indicators that directly measures the MGD Program's progress toward achieving its objectives. * Develop policies and procedures to guide evaluation of completed projects. * Formalize policies and procedures for closing out grant agreements and establish guidance for when agreements should be closed. Agency Comments and Our Evaluation: We requested and received comments on a draft of this report from the Secretary of Agriculture. The USDA agreed with our recommendations and said that it will take steps to address them. USDA acknowledged that proper monitoring and evaluation are essential to improving the quality and measuring the results of the MGD Program. For example, USDA said that it will identify a common set of indicators on which all program participants will be required to report, beginning in fiscal year 2012 program cycle, in order to permit more effective program management and an assessment of program impact, among other things. In addition, USDA said that it will finalize its monitoring and evaluation policy by September 2011. USDA also acknowledged the importance of closing out grant agreements in a timely manner to ensure full accountability of all projects and stated that it has set a goal of closing 65 percent of 2003 and 2004 agreements by September 30, 2011. Although USDA stated that it has never had a backlog of MGD Program agreements awaiting closeout, we found that 15 of 42 (36 percent) of MGD Program grants awarded before 2008 have completed program activities but have not yet been closed by USDA. USDA stated that it expects to close out agreements within 180 days of the receipt of the final reports and documents and that it will revisit its closeout procedures to seek additional improvements, including establishing timeframes for organizations to submit required documents and holding those who do not meet deadlines accountable. USDA's comments, along with our response, are reprinted in appendix VIII. USDA, USAID, and WFP also provided technical comments, which were incorporated in the report, as appropriate. We are sending copies of this report to interested members of Congress; the Secretary of Agriculture; the Administrator of the Foreign Agricultural Service; the Administrator of USAID, and relevant agency heads. The report is also available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov. If you or your staffs have any questions about this report, please contact me at (202) 512-9601 or melitot@gao.gov. Contact points for our Office of Congressional relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IX. Signed by: Thomas Melito, Director: International Affairs and Trade: List of Requesters: The Honorable Debbie Stabenow: Chairwoman: The Honorable Pat Roberts: Ranking Member: Committee on Agriculture, Nutrition, and Forestry: United States Senate: The Honorable Frank D. Lucas: Chairman: The Honorable Collin C. Peterson: Ranking Member: Committee on Agriculture: House of Representatives: The Honorable Robert P. Casey: Chairman: The Honorable Richard G. Lugar: Ranking Member: Subcommittee on Nutrition, Specialty Crops, Food and Agricultural Research: Committee on Agriculture, Nutrition, and Forestry: United States Senate: The Honorable Donald M. Payne: Ranking Member: Subcommittee on Africa, Global Health, and Human Rights: Committee on Foreign Affairs: House of Representatives: The Honorable Sherrod Brown: United States Senate: The Honorable Saxby Chambliss: United States Senate: The Honorable James P. McGovern: House of Representatives: [End of section] Appendix I: Objective, Scope, and Methodology: We examined (1) the extent to which the U.S. Department of Agriculture (USDA) is overseeing the McGovern-Dole International Food for Education and Child Nutrition Program (MGD Program) to ensure it is meeting its objectives; and (2) the extent to which USDA has addressed the MGD Program's internal control weaknesses. To examine the extent to which USDA is overseeing the MGD Program to ensure it is meeting its objectives of education, nutrition, and sustainability, we analyzed three phases of the program: (1) the grant approval phase, including agreements USDA signed with nongovernmental organizations (NGO) and the World Food Program (WFP), as well as WFP's proposals and country programs; (2) the monitoring of implementation phase, including NGO and WFP progress reports to USDA for our fieldwork countries--Cambodia, Guatemala, and Kenya, and (3) the evaluation phase, including progress in evaluating completed projects. Our discussion of the three phases considered recognized standards and principles, including USDA's criteria for designation of priority countries and proposal assessments; GAO's Standards for Internal Control in the Federal Government[Footnote 49]; Committee of Sponsoring Organizations of the Treadway Commission, Internal Control - Integrated Framework; the American Evaluation Association's Evaluation Roadmap for a More Effective Government, United Nations World Food Program, Monitoring and Evaluation Guidelines, as well as GAO's prior work in this are[Footnote 50]a. Based on standards and sources cited, GAO determined the key elements of the grant approval, monitoring, and evaluation phases of the MGD Program. To analyze the grant approval phase, we reviewed the priority country designation and solicitations for proposals (program considerations) for fiscal years 2008 through 2010, as well as the program analysts' assessment of the accepted grant proposals for fiscal year 2010. In addition, we analyzed all the MGD Program agreements that USDA's Foreign Agricultural Service (FAS) initiated with the NGOs and WFP in fiscal years 2008, 2008, and 2010. During this period 19 NGOs[Footnote 51] and 12 WFP agreements were initiated, representing funding of about $370 million. We examined the planned school feeding and take- home rations, the methods implementing partners planned to use for selecting beneficiaries, the complementary activities to achieve program goals, other donors' planned contributions, planned coordination, discussions of monitoring and evaluation, and the performance indicators selected. As WFP provided most of this information in its program proposals and country program documents, we also reviewed those documents for WFP programs. We used a data collection instrument to collect consistent information from the agreements, proposals, and country program documents, and to develop comprehensive lists of categories for each area we examined. For example, we developed more than 20 broad categories for the complementary activities and more than 40 broad categories for the targeting methods. However, we relied on the lists of more than 30 indicators that the MGD Program provided for the performance indicators. In our review, we determined how many implementing partners' programs had planned to use each category of complementary activities, targeting methods, performance indicators, and other elements of interest. In addition, we determined the average numbers of complementary activities the implementing partners' planned to use and the performance indicators they selected. Appendix V provides information on the methods NGOs agreed to use to select MGD beneficiaries. Appendix VI provides some planned complementary activities that NGOs and WFP agreed to implement. Appendix VII lists the performance indicators along with the number of NGO agreements and WFP proposals that include each indicator. To analyze the monitoring of implementation phase of the MGD Program in achieving its objectives, we reviewed the NGOs' logistics and monetization reports and project status reports, as well as WFP's standard project reports, for our fieldwork countries--Cambodia, Guatemala, and Kenya. These reports were submitted to USDA from 2004 to 2010. We did not verify the accuracy of information contained in these progress reports. We limited this analysis to our fieldwork countries because conducting fieldwork in these countries allowed us to compare the reporting of the implementing partners in these countries to our observations and the testimonial evidence provided during our site visits. The fieldwork provided context for our analysis of implementing partners' reporting. We also limited our analysis to these three fieldwork countries due to budgetary and time constraints; therefore, our findings are not generalizable to the universe of all countries where the MGD Program is implemented. Our fieldwork country selections were based on a range of criteria, including: (1) geographical dispersion, (2) amount of funding (commodity and financial assistance) recipient countries received, (3) number of beneficiaries served, (4) length of time receiving MGD Program funding, (5) diversity of the implementing partners, and (6) logistics and budget constraints. Further, to determine staff available to monitor the MGD Program, we reviewed information on staff in Washington, D.C., and overseas posts. We believe the staffing data are reasonably reliable for the purposes of this report. To compare the number of countries USDA staff visited to the total number of MGD countries, we used USDA's data on staff visits to determine the number of countries visited and USDA's expenditure data to determine the total number of MGD countries with expenditures over $100,000 for each fiscal year between 2004 and 2010. We determined that these two sets of data were sufficiently reliable for this purpose. We also determined that USDA's data on obligations by country, region, and year for fiscal years 2003 through 2010 were sufficiently reliable for our purposes. To examine USDA's evaluation phase of the MGD Program, we reviewed the MGD Program's regulations, interviewed relevant USDA officials, and analyzed evaluations of school feeding programs that had been conducted by MGD Program implementing partners. We also used the data collection instrument described above to analyze whether the 2008 through 2010 NGOs agreements and WFP proposals and country programs outlined or discussed how implementing partners would monitor and evaluate the MGD Program and whether implementing partners planned to conduct any evaluation as part of complementary activities. In addition, we considered recognized standards, such as the American Evaluation Association's Evaluation Roadmap for a More Effective Government that is intended to promote the integration of evaluation with program management. We also reviewed GAO's prior work in this area. To assess the extent to which school feeding programs can improve education and nutrition, we conducted a search for studies examining the impacts of school feeding programs in developing countries, narrowing the search to studies that evaluated the impacts of programs most similar to the MGD Program. We reviewed the resulting 21 studies for their description of the program, the study's design, methodology, limitations, and findings. The review covered studies from 1989 to 2010 (see appendix II for our review of selected studies on school feeding). To determine the extent to which USDA has implemented internal controls for the MGD Program, we reviewed the design of financial management controls in place and compared those controls to our Standards for Internal Control in the Federal Government.[Footnote 52] We then tested the implementation of selected controls. We tested all three MGD Program grants that were terminated and determined that expenditures for these grants were within budget. The results of our testing are not generalizable to the universe of all MGD grants. We also reviewed policies and procedures and interviewed financial management personnel about controls in place to ensure that cash, commodities, and transportation expenditures for MGD Program grants remain within budget. We also interviewed officials about procedures used to monitor MGD Program disbursements at the grant agreement level, and then analyzed spreadsheets being used to track these MGD Program disbursements. In particular, we traced a judgmental sample of key transactions back to the general ledger to verify the accuracy of financial information in the spreadsheets. We did not trace transactions back to source documents. We also used the spreadsheets to analyze USDA obligation and expenditure data from 2003 through 2010 to determine the extent to which NGOs have reimbursed USDA for unspent or misspent MGD Program funds. We then reviewed other financial and nonfinancial documents used by USDA officials to oversee the MGD Program, including NGO financial reports, WFP Standard Project Reports, and USDA closeout reports for our case study countries for fiscal years 2002 to 2003 and 2009. We chose these years because they represent the earliest program years and the most recently completed program fiscal year. We did not verify the accuracy of information included in these reports. We also examined NGOs A-133 audit reports for our case study countries for fiscal years 2002 to 2003 and 2009 to determine whether the USDA officials followed up on reported control problems, and whether the MGD Program had ongoing or systemic internal control problems. We then reviewed policies and procedures for closing out grant agreements, and interviewed USDA officials responsible for conducting closeouts. We assessed grant agreements awarded between 2003 and 2010 to determine their closeout status. We also reviewed policies and procedures and met with officials to determine the extent to which USDA conducts financial compliance reviews of the NGOs implementing the MGD Program. We also obtained the results of MGD Program compliance reviews from fiscal years 2005 through 2010 reported to us by USDA officials. Finally, we evaluated internal USDA assessments of financial management control weaknesses, as well as recent USDA Office of the Inspector General's audits of FAS's food aid programs to identify findings relevant to the MGD Program[Footnote 53]. We then met with USDA officials and reviewed documentation to identify the status of corrective actions taken in response to these audits. To address both our objectives, we reviewed USDA and implementing partners' program documents and financial information, and numerous studies of school feeding programs. We conducted fieldwork in Cambodia, Guatemala, and Kenya and met with representatives of WFP and NGOs who implement the program. We also met with staff from U.S. missions, host governments, the World Bank, UNICEF, and the United Nations Food and Agriculture Organization in these countries as appropriate. We visited selected schools at which the MGD projects are implemented in each country as well as food warehouses. In Washington, D.C., we interviewed officials from U.S. agencies, including USDA, the U.S. Agency for International Development (USAID), and the Departments of State and the Treasury, as well as private foundations and research institutions, such as the Bill and Melinda Gates Foundation, the Delhi School of Economics, Global Child Nutrition Foundation, International Food Policy Research Institute, and Tufts University. In Rome, we interviewed representatives of the U.S. Mission to the United Nations Food and Agriculture Agencies, WFP, the Food and Agriculture Organization, and several bilateral donors. Finally, we convened a roundtable of 9 former and current NGO implementing partners of the MGD Program and the Global Food for Education Initiative pilot to obtain their views on the program's implementation. We conducted this performance audit from May 2010 to May 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: GAO Analysis of Selected Studies on School Feeding Programs: To assess the extent to which school feeding programs can improve education and nutrition, we conducted a search for studies examining the impacts of school feeding programs in developing countries, narrowing the search to studies that evaluated the impacts of programs most similar to the MGD Program[Footnote 54]. We reviewed the resulting 21 studies for their description of the program, the study's design, methodology, limitations, and findings. The review covered studies from 1989 through 2010.[Footnote 55] Table 4 lists the results of this review. Education: Based on our review, we found school feeding programs improve attendance, especially the attendance of those students already enrolled in school before the program. However, the studies did not have consistent findings for enrollment. Five of eight studies showed improvements in enrollment with one study that found evidence that implied that students may be switching from schools without the program to schools with the program. Our review of the studies also showed mixed results on the impact of school feeding on educational progress. Although six of nine studies showed improvements in achievement and cognitive tests, the improvements were not consistent across subject matters or subpopulations. For example, one study found improvements in vocabulary test scores but not math tests scores for heavier students and another study found improvements in arithmetic test scores but not reading test scores in younger children. Nutrition: Five of five studies showed that school feeding programs increased food intake, meaning that when students were provided school feeding, households generally did not reduce the students' food intake at home enough to nullify the effect of the food they receive from school. Our review also found five studies that showed school feeding increased weight; two of these five studies showed improvements in height and other anthropometric measures. Further, there were three studies that measured micronutrient status with blood, urine, or stool testing. Of these three studies, two showed that some but not all micronutrient outcomes were improved. Sustainability: Our review of studies did not include studies that examined the sustainability of school feeding programs. Table 4: Summary of Selected Studies on School Feeding Programs: Study authors, title, publication, and year: Adelman, Sarah, Harold Alderman, Daniel O. Gilligan, and Kim Lehrer, "The Impact of School Feeding Programs on Cognitive Development and Learning: Experimental Evidence on the Role of Nutrition and Schooling from Northern Uganda," unpublished working paper, 2009; Setting: Internally displaced people camps, Uganda; Food provided: In-school feeding: snack of corn-soya blend, sugar, water and lunch of maize and beans; Take-home rations: similar in size and composition; Findings: The study used a randomized evaluation that allowed for identification of impacts on achievement and cognitive tests of in- school feeding and take-home rations, respectively. However, the study found that the impacts of the programs varied by age, subject, and statistical method, and thus the evidence on improvements in achievement and cognitive tests was mixed. Study authors, title, publication, and year: Afridi, Farzana, "The Impact of School Meals on School Participation: Evidence from Rural India," Indian Statistical Institute Discussion Paper 10-02, 2010; Setting: Poor rural district in Madhya Pradesh, India; Food provided: In-school feeding: wheat or rice porridge; Take-home rations: raw wheat or rice; Findings: The study used school level data from two different time periods and a staggered program start to compare attendance and enrollment for villages that began providing cooked meals to villages that continued to provide take-home rations. It found that switching from take-home rations to in-school feeding improved attendance for grade 1 girls. However, it found no significant increases for other grades or for boys and no significant effects on enrollment. Study authors, title, publication, and year: Afridi, Farzana, "Child Welfare Programs and Child Nutrition: Evidence from a Mandated School Meal Program in India," Journal of Development Economics 92 (2010): 152-165; Setting: Poor rural district in Madhya Pradesh, India; Food provided: In-school feeding: wheat or rice porridge; Take-home rations: raw wheat or rice; Findings: The study used household and student level data from two different time periods, a staggered program start, and dietary recall to compare the difference in food intake during school days versus non- school days and then to compare this difference among students who received in-school meals versus students who received take-home rations. The study found that in-school meals substantially improved calorie, carbohydrate, and protein intake when compared to take-home rations. The study found less consistent evidence for improvements in iron intake and almost no evidence for improvements in calcium intake. Study authors, title, publication, and year: Agarwal, DK, KN Agarwal, and SK Upadhyay, "Effect of Mid-day Meal Programme on Physical Growth and Mental Function," Indian Journal of Medical Research 90 (1989): 163-74; Setting: Rural Tamil Nadu, India; Food provided: Unspecified meals; Findings: The study randomly chose treatment schools to compare to the outcomes of control schools. However, the study did not explicitly state that the study was planned as a randomized evaluation that can identify the impacts of the program. Nonetheless, the study found that the meals improved weight, cognitive test scores, and math test scores. However, the evidence for cognitive performance differed across the statistical method of analysis and were not consistent across all cognitive tests. The study also found no evidence for improvements in height or nutritional status. Study authors, title, publication, and year: Alderman, Harold, Daniel O. Gilligan, and Kim Lehrer, "The Impact of Food for Education Programs on School Participation in Northern Uganda," unpublished working paper, 2010; Setting: Internally displaced people camps, Uganda; Food provided: In-school feeding: snack of corn-soya blend, sugar, water and lunch of maize and beans; Take-home rations: similar in size and composition; Findings: The study used a randomized evaluation that allowed for identification of impacts on attendance and enrollment of in-school feeding and take-home rations, respectively. The study found that in- school feeding and take-home rations improve attendance, age of entry, and grade repetition. The study also showed that in-school feeding increased the enrollment of out-of-school children. Finally, the study found an unintended negative impact on students' promotion rates to secondary school as hungry children may have delayed completing primary school to continue receiving the meals. Study authors, title, publication, and year: Du, Xueqin, Kun Zhu, Angelika Trube, Qian Zhang, Guansheng Ma, Xiaoqi Hu, David R. Fraser, and Heather Greenfield, "School-milk Intervention Trial Enhances Growth and Bone Mineral Accretion in Chinese Girls Aged 10-12 Years in Beijing," British Journal of Nutrition 92 (2004):159-168; Setting: Beijing, China; Food provided: Calcium milk and milk supplemented with Vitamin D; Findings: The study used a randomized evaluation that allowed for identification of impacts of providing two different kinds of milk to primary school-aged girls. The study found that providing milk to 10 year old girls significantly increased their changes in height, weight, and bone mineral content and density. The study also found that dietary recall showed increases in the intake of milk and calcium and in the intake of vitamin D for the study group that received vitamin D supplemented milk. Providing milk also led to changes in urine and blood tests for one of four micronutrient measures and providing vitamin D supplemented milk led to changes in urine and blood tests for two of four micronutrient measures. Study authors, title, publication, and year: Grillenberger, Monika, Charlotte G. Neumann, Suzzanne P. Murphy, Nimrod O. Bwibo, Pieter van't Veer, Joseph G. A. J. Hautvast, and Clive E. West, "Food Supplements Have a Positive Impact on Weight Gain and the Addition of Animal Source Foods Increases Lean Body Mass of Kenyan Schoolchildren," The Journal of Nutrition 133 (2003): 3957S-3964S; Setting: Rural area in Eastern Province, Kenya; Food provided: Githeri (maize, beans, vegetables, and fat) with meat, githeri with milk, githeri with more fat; Findings: The study used a randomized evaluation that allowed for the identification of impacts on anthropometric measures of providing meals that were supplemented with one of three different nutrients. The study found that supplements of meat, energy, or milk to the meal improved weight, middle upper arm circumference, and mid upper arm muscle area when compared to providing no food supplements. However, the study found no improvements on six other anthropometric measures, including height. Study authors, title, publication, and year: He, Fang,[A] "School Feeding Programs and Enrollment: Evidence from Sri Lanka," unpublished working paper, 2009; Setting: Sri Lanka; Food provided: Two in-school feeding programs: (1) provided corn-soya blend and (2) provided local food products; Findings: The study used school level enrollment and administrative data that was collected annually and a staggered program start to compare two differences: (1) the difference across time before and after a school meal program was implemented and (2) the difference across schools and grades with and without a school meal program. The study showed that enrollment increased for one of the school meal programs examined, but that this increase disappeared when the study looked at areas with varying intensities of program coverage. The study also found evidence that areas with higher percentages of schools and grades with the program were associated with decreases in enrollment for those schools and grades without the program in those areas. The study suggested that these results may have resulted from students switching from schools and grades without the program to schools and grades with the program. Study authors, title, publication, and year: Jacoby, Enrique, Santiago Cueto, and Ernesto Pollitt, "Benefits of a School Breakfast Programme Among Andean Children in Huaraz, Peru," Food and Nutrition Bulletin 17 No. 1 (1996): 54-64; Setting: Rural, mountainous area, Peru; Food provided: Four cookies and an instant drink (sometimes cake and drinks were provided); Findings: The study used a randomized evaluation that allowed for identification of impacts of providing cookies and an instant drink. Dietary recall was used to measure food intake. The study found that providing breakfast increased intake of energy, protein, and iron and improved attendance. However, the study did not find evidence that the program significantly improved scores on cognitive tests, reading, vocabulary, or math tests. Finally, the study found some evidence that breakfast improved vocabulary test scores for heavier students. Study authors, title, publication, and year: Jacoby, Hanan G, "Is There an Intrahousehold 'Flypaper Effect'? Evidence from a School Feeding Programme," The Economic Journal 112 (January 2002): 196-221; Setting: Metropolitan Cebu area, Phillipines; Food provided: Morning or afternoon snack (most had sweetened porridge of bulgur wheat, others had rice, vegetables, milk); Findings: The study used administrative data and student level data and dietary recall to compare the difference in food intake during school days to non-school days and then to compare this difference among students who received in-school meals to students who did not. Even though the study found that poorer households are more likely to give students less food at home in response to the school snack that was provided, overall, the study found evidence that providing a school snack increased total caloric intake. Study authors, title, publication, and year: Kazianga, Harounan, Damien de Walque, and Harold Alderman, "Educational and Child Labor Impacts of Two Food for Education Schemes: Evidence from a Randomzied Trial in Rural Burkina Faso," unpublished working paper, 2010; Setting: Sahel region (rural area), Burkina Faso; Food provided: In-school feeding: unspecified food; Take-home rations: 10 kg of cereal flour; Findings: The study used a randomized evaluation that allowed for identification of impacts of providing in-school feeding and take-home rations. The study found evidence that in-school feeding and take-home rations improved new enrollment for younger girls. The study did not find impacts on achievement or cognitive tests. The study also found a worsening overall attendance rate that could be due to the fact that new children from households with low child labor supplies (fewer siblings) were enrolled who would have been working - these new children attended school at a low rate and therefore drove down the overall attendance rate. Study authors, title, publication, and year: Meng, Xin, and Jim Ryan, "Does a Food for Education Program Affect School Outcomes? The Bangladesh Case," Journal of Population Economics 23 (2010): 415-447; Setting: Bangladesh; Food provided: Take-home rations: monthly rations of wheat (15-20 kg) or rice (12-16 kg); Findings: The study used data from a survey of schools, households, communities, and food grain dealers and matching techniques to compare four groups: (1) those that enroll into the program in program areas, (2) those that do not enroll into the program in program areas, (3) those that would have enrolled into the program in non-program areas, and (4) those that would not have enrolled into the program in non- program areas. The study found evidence that the take-home rations improved enrollment and duration of school, particularly for girls. Study authors, title, publication, and year: Murphy, Suzanne P., Constance Gewa, Li-Jung Liang, Monika Grillenberger, Nimrod O. Bwibo, and Charlotte G. Neumann, "School Snacks Containing Animal Source Foods Improve Dietary Quality for Children in Rural Kenya," The Journal of Nutrition 133 (2003): 3950S-3956S; Setting: Rural area in Eastern Province, Kenya; Food provided: Githeri (maize, beans, vegetables, and fat) with meat, githeri with milk, githeri with more fat; Findings: The study used a randomized evaluation that allowed for the identification of impacts of providing meals that were supplemented with one of three different nutrients. The study used dietary recall to measure food intake. The study found evidence that a supplement of meat to the meal improved total caloric intake. However, the study found that supplements of energy and milk did not improve total caloric intake due to decreased caloric intake at home relative to the control group. The study also found that the meat supplement increased intake of protein, vitamins B-12 and A, calcium, iron, and zinc, that the milk supplement increased intake of vitamin B-12 and A, riboflavin, and calcium, and that the energy supplement increased intake of iron and no other outcomes. Study authors, title, publication, and year: Nielsen, Nicolai Steen, Kate Godden, Pierre Leguéné, Daniela Ruegenberg, and Jesper Rüdiger, "WFP Cambodia School Feeding 2000-2010: A Mixed Method Impact Evaluation," WFP Impact Evaluation, 2010; Setting: Prey Veng and Siem Reap provinces, Cambodia; Food provided: In-school feeding: early morning meal of rice, oil, salt, fish, and yellow split pea; Take-home rations: rice, oil, salt, fish, and yellow split pea; Findings: The study used school level data collected annually to examine the effects on enrollment, promotion, continuation, and drop- out rates and cross- sectional data on students to examine the effects on attendance, achievement tests, anthropometry, and nutritional status. Control schools were chosen to be similar to program schools. The study found evidence showing improvements in enrollment, drop-out rates (3 of 6 grades), promotion (1 of 6 grades), and continuation rates (1 of 6 grades). The study also used cross-sectional data to find evidence that take-home rations improved attendance and that in- school feeding increased girls' weight, but the study did not find significant evidence showing increases in girls' height and no evidence showing increases in girls' mid upper arm circumference. The study did not find any evidence showing increases in boys' anthropometric measures. The only evidence that the study found for improving achievement test scores was for girls' math tests scores in one of the two provinces. The study found that the improvements in enrollment and attendance were stronger for girls. The study also found evidence on reducing illness for girls and on improving dietary diversity for boys and girls. The study did not find that school feeding improved hemoglobin levels or reduced vitamin A deficiencies significantly. Study authors, title, publication, and year: Powell, Christine A., Susan P. Walker, Susan M. Chang, and Sally M. Grantham-McGregor, "Nutrition and Education: A Randomized Trial of the Effects of Breakfast in Rural Primary School Children," The American Journal of Clinical Nutrition 68 (1998): 873-9; Setting: Rural, mountainous area, Jamaica; Food provided: Breakfast (cheese sandwich or spiced bun and cheese and flavored milk), orange slice; Findings: The study used a randomized evaluation that allowed for the identification of impacts of providing breakfast versus providing an orange slice. The study found evidence showing that providing breakfast improved attendance, height, weight, and body mass index. As for the achievement tests, the study found that breakfast improved arithmetic test scores for young children but not spelling or reading test scores. Study authors, title, publication, and year: Ravallion, Martin and Quentin Wodon, "Evaluating a Targeted Social Program When Placement is Decentralized," unpublished working paper, 1998; Setting: Rural Bangladesh; Food provided: Take-home rations: monthly rations of wheat or rice 15 kg for one child, 30 kg for more than two children in household; Findings: The study used household level data and implemented a methodology to address the likely bias in estimating the program's impact that was a result of differences between households that participate in the program and households that do not. Thus the study used an instrumental variables methodology with the program area as the instrument since the selection of the program (geographic) area was based on specified area characteristics. The study used this methodology to find evidence that showed that take-home rations improved attendance rates. Study authors, title, publication, and year: Ravallion, Martin and Quentin Wodon, "Does Child Labour Displace Schooling? Evidence on Behavioural Responses to an Enrollment Subsidy," The Economic Journal 110 No. 462 (March 2000): C158-C175; Setting: Rural Bangladesh; Food provided: Take-home rations: monthly rations of rice that total on average 114 kg per year per household; Findings: The study used household level data and implemented a methodology to address the likely bias in estimating the program's impact that was a result of differences between households that participate in the program and households that do not. Thus the study used an instrumental variables methodology with the program area as the instrument since the selection of the program (geographic) area was based on specified area characteristics. The study used this methodology to find evidence that showed that take-home rations improved enrollment and reduced child labor. However, only part of the increase in enrollment was accounted for by a reduction in child labor. Study authors, title, publication, and year: Rogers, Beatrice Lorge, Jennifer Coates, and Akoto Kwame Osei, "WFP Bangladesh School Feeding Programme Mid-term Evaluation Final Report," unpublished working paper, 2004; Setting: Rural areas and urban slums, Bangladesh; Food provided: Biscuits; Findings: The study used school level data from two time periods to compare two differences: (1) the difference across time before and after take-home rations were provided and (2) the difference between those that received the take-home rations and those that did not. The study found evidence, albeit not consistent across all statistical methods, that take-home rations increased attendance. However, the study found weak or no evidence that take-home rations improved enrollment, grade repetition, drop-out rates, or achievement tests. The study did find that the enrollment at schools without the program decreased during the study period. Study authors, title, publication, and year: Siekmann, Jonathan H., Lindsay H. Allen, Nimrod O. Bwibo, Montague W. Demment, Suzanne P. Murphy, and Charlotte G. Neumann, "Kenyan School Children Have Multiple Micronutrient Deficiencies, but Increased Plasma Vitamin B-12 is the Only Detectable Micronutrient Response to Meat or Milk Supplementation," The Journal of Nutrition 133 (2003): 3972S-3980S; Setting: Rural area in Eastern Province, Kenya; Food provided: Githeri (maize, beans, vegetables, and fat) with meat, githeri with milk, githeri with more fat; Findings: The study used a randomized evaluation that allowed for the identification of impacts of providing meals that were supplemented with one of three different nutrients. The study used clinical exams, blood tests, and stool samples to assess the impacts on micronutrient outcomes and disease prevalence rates. The study found evidence that showed that 1 of 9 micronutrient outcomes (plasma vitamin B-12) were improved for supplements of meat and milk to the meal. The study did not find evidence that a supplement of energy to the meal improved micronutrient outcomes. The study also showed that the meat supplement improved prevalence rates for 5 of 10 diseases, the milk supplement improved prevalence rates for 2 of 10 diseases, and the energy supplement improved prevalence rates of 1 of 10 diseases. Study authors, title, publication, and year: Tan, Jee-Peng, Julia Lane, and Gerard Lassibille, "Student Outcomes in Philippine Elementary Schools: An Evaluation of Four Experiments," The World Bank Economic Review 13 No. 3 (September 1999): 493-508; Setting: 10 provinces, Philippines; Food provided: Unspecified meals; Findings: The study used a randomized evaluation that allowed for the identification of impacts of providing school feeding compared to providing learning materials and providing nothing. The school feeding and learning materials program was also paired with providing parent- teacher partnerships. The study also looked at the cost of the different programs. However, we focused on the study's findings on school feeding since that is the focus of this review. The study did not find evidence that school feeding improved the drop-out rate from 1st to 5th grade. The study also did not find consistent evidence (across different model specifications) of improvements in math, Filipino, or English achievement test scores. Finally, despite being a randomized evaluation, the study had differences between the study groups that led to different results for different statistical models. Study authors, title, publication, and year: Whaley, Shannon E., Marian Sigman, Charlotte Neumann, Nimrod Bwibo, Donald Guthrie, Robert E. Weiss, Susan Alber, and Suzanne P. Murphy, "The Impact of Dietary Intervention on the Cognitive Development of Kenya School Children," The Journal of Nutrition 133 (2003): 3965S-3971S; Setting: Rural area in Eastern Province, Kenya; Food provided: Githeri (maize, beans, vegetables, and fat) with meat, githeri with milk, githeri with more fat; Findings: The study used a randomized evaluation that allowed for the identification of impacts on cognitive test scores of providing meals that were supplemented with one of three different nutrients. The study found evidence that showed that providing a supplement of meat to the meal improved scores on two of three cognitive tests (Raven's matrices and arithmetic). The study also showed that providing a supplement of energy to the meal improved arithmetic test scores. However, the study did not find that providing a supplement of milk to the meal improved any cognitive test scores. Source: GAO analysis of selected studies on school feeding. [A] Fang He, an economist at GAO, completed this study prior to his current employment at GAO. His preparation of the summary of selected studies on school feeding programs was independently reviewed by two other GAO economists who concurred with the analysis presented here. [End of table] [End of section] Appendix III: Operation of the MGD Program: The MGD Program Grant Approval Process Begins with the Designation of Priority Countries: USDA has established a grant approval process to help ensure that only programs that are likely to achieve the MGD Program's objectives of education, nutrition, and sustainability are approved. As such, USDA has developed criteria for designating priority countries, evaluating proposals, and negotiating grant agreements. Annually, according to FAS, it selects the MGD Program's priority countries that would most benefit from U.S. food assistance based on criteria that are intended to help ensure that the priority countries are needy and are committed to improving their education systems. Table 5 lists the criteria that a country had to meet to be considered a priority country. In order to help ensure sustainability, countries with existing programs were also considered priority countries. Table 5: Criteria Used to Determine Priority Countries: Fiscal Year 2008: Per capita incomes below $3,465 (based on World Bank statistics); Fiscal Years 2009 and 2010: Per capita income below $3,595 (based on World Bank statistics) and a population greater than 1 million; Fiscal Year 2011: Low-and lower-middle income countries (based on 2008 World Bank statistics). Fiscal Year 2008: A net food importer with a greater-than-20-percent prevalence of undernourishment as a proportion of the total population (Food and Agricultural Organization); Fiscal Years 2009 and 2010: A net food importer with a greater-than-20- percent prevalence of undernourishment as a proportion of the total population (Food and Agricultural Organization); Fiscal Year 2011: Greater than 20-percent prevalence of stunted growth (World Health Organization). Fiscal Year 2008: Adult literacy rate below 75 percent; Fiscal Years 2009 and 2010: Adult literacy rate below 75 percent; Fiscal Year 2011: Adult literacy rate below 80 percent. Fiscal Year 2008: Government support for education; Fiscal Years 2009 and 2010: Government commitment for education; Fiscal Year 2011: Government support for education. Fiscal Year 2008: Has no or limited civil conflict that could impede implementation of the program; Fiscal Years 2009 and 2010: No security, market, and/or capacity issues exist; Fiscal Year 2011: Absence of civil conflict. Fiscal Years 2009 and 2010: Fiscal Year 2008Fiscal Years 2009 and 2010: FAS post coverage and FAS ability to monitor program implementation; Fiscal Year 2011: Fiscal Year 2008Fiscal Year 2011: FAS overseas presence to provide oversight of program activities. Source: GAO analyses of FAS's program considerations (solicitations for proposals) for fiscal years 2008 through 2011. Note: In fiscal year 2009, FAS solicited proposals for two fiscal years: 2009 and 2010. [End of table] Entities Submit Proposals Based on USDA's Criteria: Once USDA identifies and announces priority countries, it invites NGOs, foreign governments, and international organizations, such as WFP, to submit proposals. The proposals are evaluated based on criteria that are designed to address the MGD Program's three broad objectives of education, nutrition, and sustainability. Food Assistance Division analysts review every proposal and evaluate them by assessing the quality of the proposal, such as how the applicants plan to implement the program to achieve its goals, the applicants' organizational capabilities, the project's graduation or sustainability plan, the appropriateness of the requested commodities for the beneficiaries, and the need for the program. Table 6 describes the criteria the Food Assistance Division used to assess each proposal and the weight assigned each criteria for fiscal years 2008 to 2010. Table 6: Criteria Used to Assess Proposals, Fiscal Years 2008-2010: Proposal criteria: Proposal quality; This section addresses three program issues: implementation, costs, and situational analysis: 1. How will the applicant implement the program, specifically, will the applicant contribute its own or outside resources to achieve program goals? How will the applicant use program funds to implement and monitor the target goals of the program? How does the proposed commodity provide a food for targeted beneficiaries that augments calories or nutrients? How will monetization or barter be conducted and by whom, and is monetization or barter more appropriate than a cash outlay? Has the applicant addressed the customs exemption issue in the case of a direct feed program? 2. Does the proposal indicate a cost ratio that can effectively achieve a direct feeding, education, or maternal health care program? Is the proposal cost efficient in reaching a large number of recipients with the requested resources? If USDA resources are requested, do these funds effectively support an experienced management team which can implement, evaluate, and monitor projected goals? Are the requested USDA funds appropriate for the proposed technical assistance, storage, distribution, or other pertinent program activities? 3. Does the proposal provide country information that explains and justifies the need for the proposed food aid programming? Do the targeted beneficiaries have a need that is not being met currently or sufficiently? Are there clear criteria and a persuasive rationale for the selection of a particular region, country and beneficiaries? Are the beneficiary baseline and target goals clear? Does the proposal target areas of a recipient country with the highest levels of poverty, hunger and low primary school enrollment rates, particularly of girls? Are the ideas in the proposal well developed and articulated? Weight: 38 percent. Proposal criteria: Experience and organizational capacity factors; Does the proposal show the organization's capability and effectiveness in implementing previous food aid programs, particularly school feeding, maternal child health (MCH), or other developmental activities related to education in schools or MCH? FAS looks at the experience of the organization and evaluates the organization favorably if it has experience in providing food aid with its own resources, or, more importantly, those from other donors. FAS considers the experiences of providing food aid regardless of the source of funding. Past experiences with USDA, USAID, or other donors will be viewed positively. FAS also reviews lists of known terrorists to ensure no organization, nor recipient agency, is participating in or funding terrorist activities. A review of non-profit websites is also conducted to ensure the financial and technical capability of program applicants. Finally, FAS ensures that organizations new to the program have a fair chance in competing for funds; Weight: 20 percent. Proposal criteria: Graduation/Sustainability--Country, community, or other donor contribution to program; Is there a plan for the host or local government to provide sufficient space and teachers for targeted schools? Does the local community provide food, space, labor, and involvement in education or nutrition programs? Does the applicant commit resources that demonstrate a continuance of the program after FAS ends funding? Does the proposal contain a graduation plan with methods and timeline to sustain activities and achievements? Do governments or other donors provide financial or in-kind support of the proposed activity that helps continue the program beyond the years of the proposed program? Does the proposal describe what other stakeholders are doing to address poverty, hunger and deficient primary education in the recipient country, what needs remain, and how the proposed program complements and does not duplicate those activities? Weight: 15 percent. Proposal criteria: Commodity or USDA funds appropriateness; Has the organization identified commodities and tonnages appropriate for the country? Is adequate information provided about the distribution process, storage and handling of commodities? Has the organization clearly identified how the requested USDA funds will be used and how they will complement the distribution of the commodities and lead to sustainability? Weight: 15 percent. Proposal criteria: Need for program; Is the program need clearly substantiated with statistics on food deficits, malnutrition, literacy, and information regarding education resources? Does the recipient country demonstrate commitment to improving its quality of education and nutrition of school-aged or children under 5 years old? Weight: 12 percent. Source: FAS's Fiscal Years 2008 through 2010 Program Considerations (solicitation for proposals). [End of table] Once USDA makes the final decisions on the proposals, the proposals are sent to the appropriate FAS overseas office for review and shared with other U.S. agencies, such as USAID, the U.S. State Department, and the Office of Management and Budget. After the awards are made, the agreements are negotiated with the implementing partners. Implementing Partners Are Required to Submit Various Progress Reports to Monitor and Evaluate the MGD Program: Food Assistance Division analysts monitor and evaluate the MGD Program's projects while they are active. The implementing partners are required to provide several periodic progress reports to FAS. Table 7 lists the types of reports, examples of information required, frequency, and party responsible for preparing them. Table 7: Periodic Progress Reporting Required of Implementing Partners: Required reports: Financial report; Examples of information required: The NGO report provides amounts for administrative expenses, such as salaries and office rent, and spending on shipping, handling, and warehousing of the commodities; Frequency of required reports: At least annually but not more than quarterly; Quarterly; Implementing partner: NGO; The WFP report will provide information regarding expenditures related to a project agreement, such as cash disbursements, federal share of expenditures, and recipient share of expenditures; Frequency of required reports: Quarterly; Implementing partner: WFP will comply beginning in fiscal year 2012. Required reports: Logistics and monetization report; Examples of information required: Examples of information covered in the report: * the commodities, the amounts received, when received, the commodity balance; * commodity losses or damages, and problems encountered in the delivery, and warehousing of the commodity; * the amount of commodity distributed to each project and the number of beneficiaries served; * school performance by several performance indicators, including enrollment and attendance; * information on monitoring procedures; and; * if applicable, the amount of food aid monetized, the amount generated from the sale of the commodity, and the use of the funds; Frequency of required reports: Semiannual; Implementing partner: NGO. Required reports: Project status report; Examples of information required: Examples of information covered in the report: * the number, frequency, and average size of the meal provided each month for in-school meals and take-home rations; * monthly enrollment, dropout, and attendance by gender for each school; * the establishment of any school infrastructure; and; * the establishment of any parent-teacher associations; Frequency of required reports: Semiannual; Implementing partner: NGO; WFP will comply beginning in fiscal year 2012. Required reports: Single Audit (A-133); Examples of information required: NGOs that expend $500,000 or more in federal awards in a year are to obtain an audit in accordance with the requirements of the Single Audit Act (codified at 311 U.S.C. 7501 et seq). A Single Audit consists of "(1) an audit and opinions on the fair presentation of the financial statements and the Schedule of Expenditures of Federal Awards; (2) gaining an understanding of and testing internal control over financial reporting and the entity's compliance with laws, regulations, and contract or grant provisions that have a direct and material effect on certain federal programs (i.e., the program requirements); and (3) an audit and an opinion on compliance with applicable program requirements for certain federal programs;" Frequency of required reports: Annual; Implementing partner: NGO. Required reports: Standard project report; Examples of information required: WFP provides a standard report that covers its country projects, including school feeding activities. Examples of the information covered in the report are: * the operational objectives of the country program; * the project's results, including beneficiaries targeting, outputs, outcomes, sustainability, and capacity development and handover; * inputs, including resources from donors, governments, and partners; food purchases, and food transport, delivery, and handling; * management, including WFP's partnerships with other entities and lessons learned; and; * the financial section provides details of financial activity relating to the project throughout the reporting period and a summary of the project's financial status at the end of the reporting period; Frequency of required reports: Annual; Implementing partner: WFP. Required reports: Ad hoc reporting; Examples of information required: For each project agreement, WFP will be required to report on collected, established baseline indicators about: * the number of meals served; * access to, entry into and continuation of school (e.g., enrollment levels, total attendance numbers, including female attendance levels); * educational progress; * nutrition and child health interventions; * other donor support; * community development; and; * sustainability for activities supported by the project agreement; Frequency of required reports: Annual; Implementing partner: WFP will comply beginning in fiscal year 2012. Required reports: Interim and final evaluations; Examples of information required: Implementing partners are required to have evaluations conducted by an independent third party that: * is financially and legally separate from the participant's organization; * has staff with demonstrated knowledge, analytical capability, language skills, and experience in conducting evaluation of development programs involving agriculture, education, and nutrition; * uses acceptable analytical frameworks such as comparison with non- project areas, surveys, involvement of stakeholders in the evaluation, and statistical analyses; * uses local consultants, as appropriate, to conduct portions of the evaluation; and; * provides a detailed outline of the evaluation, major tasks, and specific schedules prior to initiating the evaluation; Frequency of required reports: Midpoint and end point of the project implementation period; Implementing partner: NGO and WFP. Sources: USDA's Foreign Agricultural Service and GAO analysis. Notes: The semiannual reports are due in November and May of each year that the project is active. WFP currently provides the annual standard project report to FAS. Based on a framework agreement between FAS and WFP signed in June 2010, WFP is required to comply with the other reporting requirements beginning in 2012. Implementing partners are expected to submit the first round of mid- point evaluations for multi-year agreements in fiscal year 2012 and the first round of end-point evaluations in fiscal year 2014. [End of table] FAS Staff Perform Closeouts of Completed Grant Agreements: FAS staff are required to closeout grant agreements after the MGD project activities are completed. Both the Food Assistance Division and the Monitoring and Evaluation Staff analysts participate in grant closeouts. Using a checklist, the Food Assistance Division program analysts compile a folder for the Monitoring and Evaluation staff that consists of the necessary documents, including final logistics and monetization reports, cumulative financial reports, and final audits. The Monitoring and Evaluation Staff first conducts a financial analysis by comparing the reported expenditures to the budgeted amounts in the agreement. The analysis consists of determining whether money is owed and ensuring that line item expenditures did not exceed the agreement, etc. The logistics and monetization reports are reviewed to assess whether excessive loss in commodities occurred, and results of the A- 133 audit are reviewed to determine the financial integrity of the NGOs. If expenditure reports indicate that all advanced cash has not been used, the NGO is required to reimburse USDA. USDA occasionally conducts financial compliance review site visits of USDA food aid grants to verify that the funds were spent as approved by USDA. Site visits are conducted by the Compliance, Security, and Emergency Planning Division within USDA. [End of section] Appendix IV: MGD Program Funding by Year, Country, and Region: The total amount of funds appropriated by Congress for the MGD Program from fiscal years 2003 through 2010 is $931 million.[Footnote 56] Table 8 shows that, based on USDA's data, the total amount obligated for MGD projects through fiscal year 2010 is approximately $892 million. Of that amount, approximately $638 million has been spent as of October 31, 2010. Of the total obligated, the five countries that received the most funding are Guatemala ($74.6 million), Pakistan ($67.3 million), Kenya ($62.6 million), Mozambique ($51.1 million), and Afghanistan ($50.7 million). Table 8: MGD Program Obligations by Year, Country, and Region, Fiscal Years 2003 through 2010 (in millions): Country: Angola; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: $29.5; 2010: [Empty]; Grand total: $29.5. Country: Benin; 2003: $5.4; 2004: [Empty]; 2005: [Empty]; 2006: $2.9; 2007: $1.7; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $9.9. Country: Cameroon; 2003: $1.6; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: $3.9; 2009: $2.9; 2010: $2.7; Grand total: $11.1. Country: Chad; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: $5.1; 2007: [Empty]; 2008: $5.6; 2009: $5.6; 2010: $5.6; Grand total: $22.0. Country: Republic of Congo; 2003: [Empty]; 2004: $2.5; 2005: [Empty]; 2006: $7.1; 2007: $6.9; 2008: $4.7; 2009: [Empty]; 2010: $3.6; Grand total: $24.8. Country: Eritrea; 2003: [Empty]; 2004: $4.5; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $4.5. Country: Ethiopia; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: $4.5; 2009: $4.5; 2010: $4.5; Grand total: $13.4. Country: Gambia; 2003: [Empty]; 2004: $0.5; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $0.5. Country: Ghana; 2003: $4.4; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $4.4. Country: Guinea; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: $11.4; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $11.4. Country: Guinea-Bissau; 2003: [Empty]; 2004: [Empty]; 2005: $2.1; 2006: $3.2; 2007: $5.0; 2008: [Empty]; 2009: $18.8; 2010: $0.1; Grand total: $29.1. Country: Ivory Coast; 2003: $4.7; 2004: $3.1; 2005: $4.7; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $12.5. Country: Kenya; 2003: [Empty]; 2004: $5.3; 2005: $10.3; 2006: $7.7; 2007: $10.0; 2008: $10.0; 2009: $10.0; 2010: $9.4; Grand total: $62.6. Country: Liberia; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: $2.3; 2008: [Empty]; 2009: $9.0; 2010: $11.4; Grand total: $22.8. Country: Madagascar; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: $5.4; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $5.4. Country: Malawi; 2003: $4.2; 2004: $3.4; 2005: [Empty]; 2006: [Empty]; 2007: $6.5; 2008: $6.5; 2009: $6.5; 2010: $8.3; Grand total: $35.6. Country: Mali; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: $14.0; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $14.0. Country: Mozambique; 2003: $3.3; 2004: $3.5; 2005: [Empty]; 2006: [Empty]; 2007: $5.4; 2008: $7.1; 2009: $7.8; 2010: $24.0; Grand total: $51.1. Country: Niger; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: $13.2; 2010: [Empty]; Grand total: $13.2. Country: Rwanda; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: $9.1; 2009: $9.1; 2010: $9.1; Grand total: $27.3. Country: Senegal; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: $7.3; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $7.3. Country: Sierra Leone; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: $2.9; 2009: $2.9; 2010: $2.3; Grand total: $8.1. Country: Tanzania; 2003: $3.9; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: $9.7; Grand total: $13.6. Country: Uganda; 2003: $2.5; 2004: [Empty]; 2005: $6.8; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: $19.0; 2010: $12.7; Grand total: $40.9. Country: Africa total; 2003: $30.0; 2004: $22.8; 2005: $23.9; 2006: $33.3; 2007: $68.5; 2008: $54.4; 2009: $138.7; 2010: $103.5; Grand total: $475.1. Country: Afghanistan; 2003: $9.0; 2004: $6.7; 2005: $10.0; 2006: $25.0; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $50.7. Country: Bangladesh; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: $8.1; 2007: [Empty]; 2008: $7.8; 2009: $7.9; 2010: $8.0; Grand total: $31.8. Country: Bhutan; 2003: $1.0; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $1.0. Country: Cambodia; 2003: $1.3; 2004: [Empty]; 2005: [Empty]; 2006: $5.0; 2007: $8.8; 2008: $1.3; 2009: $1.2; 2010: $18.1; Grand total: $35.9. Country: Kyrgyzstan; 2003: $3.0; 2004: $3.0; 2005: $6.6; 2006: [Empty]; 2007: [Empty]; 2008: $2.4; 2009: [Empty]; 2010: $3.2; Grand total: $18.2. Country: Laos; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: $3.3; 2007: [Empty]; 2008: $6.9; 2009: $3.0; 2010: $7.3; Grand total: $20.5. Country: Nepal; 2003: $5.4; 2004: [Empty]; 2005: $6.2; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $11.6. Country: Pakistan; 2003: $6.9; 2004: [Empty]; 2005: $7.5; 2006: $7.3; 2007: $9.5; 2008: $9.9; 2009: $10.0; 2010: $16.3; Grand total: $67.3. Country: Vietnam; 2003: $8.0; 2004: [Empty]; 2005: $6.8; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $14.8. Country: Asia total; 2003: $34.7; 2004: $9.7; 2005: $37.1; 2006: $48.8; 2007: $18.3; 2008: $28.3; 2009: $22.1; 2010: $52.9; Grand total: $251.8. Country: Albania; 2003: $3.2; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $3.2. Country: Moldova; 2003: $9.0; 2004: $6.6; 2005: $4.6; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $20.2. Country: Eastern Europe total; 2003: $12.2; 2004: $6.6; 2005: $4.6; 2006: $0.0; 2007: $0.0; 2008: $0.0; 2009: $0.0; 2010: $0.0; Grand total: $23.4. Country: Bolivia; 2003: $3.4; 2004: $3.0; 2005: $3.7; 2006: $1.8; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: $7.0; Grand total: $18.9. Country: Dominican Republic; 2003: [Empty]; 2004: $3.4; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $3.4. Country: Guatemala; 2003: $4.8; 2004: [Empty]; 2005: $4.0; 2006: $10.5; 2007: $11.2; 2008: $13.7; 2009: $14.2; 2010: $16.1; Grand total: $74.6. Country: Honduras; 2003: [Empty]; 2004: [Empty]; 2005: $9.7; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $9.7. Country: Nicaragua; 2003: $5.4; 2004: [Empty]; 2005: $2.7; 2006: $0.7; 2007: [Empty]; 2008: $1.2; 2009: [Empty]; 2010: $0.1; Grand total: $10.1. Country: Latin America and the Caribbean total; 2003: $13.6; 2004: $6.4; 2005: $20.2; 2006: $12.9; 2007: $11.2; 2008: $14.9; 2009: $14.2; 2010: $23.2; Grand total: $116.7. Country: Lebanon; 2003: $7.9; 2004: [Empty]; 2005: $7.8; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: [Empty]; 2010: [Empty]; Grand total: $15.7. Country: Yemen; 2003: [Empty]; 2004: [Empty]; 2005: [Empty]; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: $9.1; 2010: [Empty]; Grand total: $9.1. Country: Middle East total; 2003: $7.9; 2004: [Empty]; 2005: $7.8; 2006: [Empty]; 2007: [Empty]; 2008: [Empty]; 2009: $9.1; 2010: [Empty]; Grand total: $24.8. Country: Grand total; 2003: $98.4; 2004: $45.5; 2005: $93.6; 2006: $95.0; 2007: $98.0; 2008: $97.6; 2009: $184.1; 2010: $179.6; Grand total: $891.8. Source: USDA, Farm Service Agency. Note: The funds for Yemen were obligated, but the MGD project was never implemented. [End of table] Of the total funds obligated, according to USDA, Africa received the most, 53 percent, followed by Asia, Latin America and the Caribbean, Middle East, and Eastern Europe, as reflected in figure 9. Figure 9: MGD Program Funds Obligated by Region, Fiscal Years 2003 Through 2010: [Refer to PDF for image: pie-chart] Africa: $475.1 million (53.3%); Asia: $251.8 million (28.2%); Latin America and the Caribbean: $116.7 million (13.1%); Middle East: $24.8 million (2.8%); Eastern Europe: $23.4 million (2.6%). Sources: USDA, Farm Service Agency. [End of figure] [End of section] Appendix V: Methods NGOs Planned to Use to Select MGD Program Beneficiaries: We analyzed all 19 NGO agreements from fiscal years 2008 through 2010 to determine how NGOs select their beneficiaries for the MGD Program. Table 9 shows the methods NGOs agreed to use in selecting their beneficiaries and the number of NGO agreements that include each method. Some of the MGD program agreements for the NGOs planned to select schools that required new facilities, teacher training, or the development of parent teacher associations or school management committees, while others planned to select schools that already had existing facilities or community organizations. According to WFP, its school feeding programs target food insecure areas, which tend to be rural, and identified through food security assessments. Food security information is combined with the government-provided education-related information (such as enrollment, attendance, retention, and completion rates). WFP has been including educational indicators into its food security assessments to inform its decisions on school feeding and is now trying to standardize this process by using a set of targeting guidelines specific for school feeding. Table 9: Methods NGOs Planned to Use to Select MGD Program Beneficiaries: Opinions of need: NGOs' methods of selecting beneficiaries: Host government agencies opinions' of need; Number of agreements/proposals: 4. NGOs' methods of selecting beneficiaries: Local teachers and education officials opinions' of need; Number of agreements/proposals: 2. NGOs' methods of selecting beneficiaries: Local provincial and regional officials opinions' of need; Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: Village elders or other community representatives' opinions of need; Number of agreements/proposals: 1. Data collection to indicate need: NGOs' methods of selecting beneficiaries: On-site observation and interviews to determine needs; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Field surveys conducted by NGO or others to determine needs (including lists created for earlier programs); Number of agreements/proposals: 8. NGOs' methods of selecting beneficiaries: Government registries of educational institutions in a region; Number of agreements/proposals: 4. NGOs' methods of selecting beneficiaries: Official host government statistics; Number of agreements/proposals: 13. NGOs' methods of selecting beneficiaries: WFP, other United Nations agency or other donor's (such as USAID) statistics; Number of agreements/proposals: 7. Indicators of need: NGOs' methods of selecting beneficiaries: High rates of poverty; Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: Low levels of literacy; Number of agreements/proposals: 3. NGOs' methods of selecting beneficiaries: High rates of malnutrition; Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: High levels of food insecurity and vulnerability; Number of agreements/proposals: 6. NGOs' methods of selecting beneficiaries: Low levels of school enrollment/underenrollment; Number of agreements/proposals: 7. NGOs' methods of selecting beneficiaries: Low levels of school enrollment for girls; Number of agreements/proposals: 3. NGOs' methods of selecting beneficiaries: High rates of absenteeism from school/low rates of attendance; Number of agreements/proposals: 4. NGOs' methods of selecting beneficiaries: Low retention rates; Number of agreements/proposals: 2. NGOs' methods of selecting beneficiaries: High drop out rates for girls, esp. between primary and secondary school; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Other evidence of gender inequality; Number of agreements/proposals: 2. Location: NGOs' methods of selecting beneficiaries: Rural or remote location; Number of agreements/proposals: 3. NGOs' methods of selecting beneficiaries: Regions not receiving coverage by other school feeding program; Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: Area(s) where NGO works; Number of agreements/proposals: 3. Community willingness to participate and contribute: NGOs' methods of selecting beneficiaries: Community agrees to participate and contribute; Number of agreements/proposals: 9. NGOs' methods of selecting beneficiaries: Community agrees to provide firewood and cooking fuel; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Local government willingness to contribute financially; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Parent Teacher Association (PTA) or School Management Committee (SMC) indicates willingness to participate and contribute; Number of agreements/proposals: 7. School willingness to participate and contribute: NGOs' methods of selecting beneficiaries: Teachers agree to participate and/or contribute (including by participating in teacher training by the NGO and contributing time to distribution activities); Number of agreements/proposals: 2. NGOs' methods of selecting beneficiaries: Schools demonstrate or agree to adhere to program policies and conditions (including regulations for storage and handling, distribution, data collection, monitoring and tracking by the NGO, reporting, etc.); Number of agreements/proposals: 4. NGOs' methods of selecting beneficiaries: Schools participate or agree to participate in related NGO programs; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Schools willing to adopt innovative practices related to sustainability; Number of agreements/proposals: 7. School can demonstrate necessary capacity: NGOs' methods of selecting beneficiaries: Schools have staff capacity and experience in school feeding programs (e.g., sufficient teachers and support staff, staff with experience in school feeding programs and management); Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Schools have necessary facilities (e.g., trucks, safe water supply, functioning kitchens, secure storage, disposal facilities, etc.); Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: PTA or SMC or Education Development Committee exists and is capable of helping run program; Number of agreements/proposals: 5. NGOs' methods of selecting beneficiaries: Schools participated in previous school feeding program; Number of agreements/proposals: 6. Other: NGOs' methods of selecting beneficiaries: Schools currently closed but could be re-opened to meet local needs; Number of agreements/proposals: 1. NGOs' methods of selecting beneficiaries: Other; Number of agreements/proposals: 25. Source: GAO analysis of NGO agreements from 2008 through 2010. [End of table] [End of section] Appendix VI: MGD Program's Planned Complementary Activities in Agreements with Implementing Partners: We analyzed all 19 NGO agreements and all 12 WFP proposals and country programs from fiscal years 2008 through 2010 to determine the types of activities that implementing partners agreed to implement in their agreements with USDA. The implementing partners agreed to these activities as part of the MGD Program's objectives to improve education, nutrition, and sustainability. Table 10 shows some planned complementary activities that NGOs and WFP have agreed to implement and the number of NGO agreements and WFP proposals and country programs that include each activity. Table 10: MGD Program's Planned Complementary Activities in Agreements with Implementing Partners: Education-related activities: Activity name: Teacher Training; Description of activities: Train teachers to improve education quality; includes training trainers, working with the host government Ministry of Education on teacher training curriculum and issues; advanced teacher training; training the trainers; Number of agreements/proposals: 14. Activity name: School Facilities: General; Description of activities: Improve school infrastructure, facilities, or environment through construction projects or grants (includes furniture, classrooms, storage rooms, and canteens--but not latrines or wells, see below); Number of agreements/proposals: 13. Activity name: Educational Supplies/Equipment; Description of activities: Provide school supplies and educational materials (including school or classroom kits); Number of agreements/proposals: 12. Activity name: School Awareness; Description of activities: Conduct school awareness campaigns and/or training emphasizing the value of education; Number of agreements/proposals: 7. Activity name: Teacher Residences; Description of activities: Provide or rehabilitate residences for teachers; Number of agreements/proposals: 2. Nutrition-related activities: Activity name: Health/Nutrition Education; Description of activities: Provide health, nutrition, hygiene sessions, and/or advice in schools and the community. Includes providing health advice/education/training for parents (esp. mothers), children (school and preschool), teachers, caregivers, others in the community; training municipal government, parents on health, nutrition, hygiene, etc. Also includes advice on preparation of food; Number of agreements/proposals: 22. Activity name: School Facilities: Potable water; Description of activities: Drill and/or rehabilitate boreholes and water wells, or provide other methods (and training) to ensure potable water; Number of agreements/proposals: 16. Activity name: School Facilities: Latrines; Description of activities: Construct school latrines; Number of agreements/proposals: 14. Activity name: Health: De-worming; Description of activities: Distribute and administer deworming medication either by WFP/NGO itself of in collaboration with other donors, such as UNICEF or local health officials; Number of agreements/proposals: 14. Activity name: Health Supplies; Description of activities: Provide health supplies (such as medicines) and supplies/equipment to promote health; Number of agreements/proposals: 7. Activity name: Nutrition Interventions; Description of activities: Provide rations and/or nutrients to mothers/families or others; distribute micronutrients and/or vitamin supplements to schoolchildren (and pregnant nursing mothers); purchase salt or other commodities locally to add to the rations; Number of agreements/proposals: 6. Activity name: Health Screening; Description of activities: Screen children for ailments/perform simple examinations and follow up as appropriate; perform simple interventions; Number of agreements/proposals: 3. Activity name: Health: Anti-malaria; Description of activities: Conduct antimalaria programs (informational materials, provision of nets and sprays); Number of agreements/proposals: 1. Sustainability-related activities: Activity name: Sustainability: Local Level; Description of activities: Train local government employees, community planners, parent-teacher associations, and other stakeholders to focus on sustainability. Includes, training parents to help with educational activities; training teachers, community leaders, and parent-teacher associations or school management committees to implement project activities; training in budget management techniques possibly with special reference to school feeding programs; adult literacy courses for local community; training in farming techniques; Number of agreements/proposals: 17. Activity name: Parent and Community Involvement in the Schools; Description of activities: Form or develop partnerships with parents and the community/create or support parent-teacher associations or school management committees; Number of agreements/proposals: 17. Activity name: Sustainability: Host Governments; Description of activities: Work with host government Ministry of Education to develop national school feeding program. Includes: working with host government Ministry of Education and Ministry of Health to develop their capacity to run and monitor education and health programs; encouraging/supporting changes in the education system, establishing national education management information system, developing monitoring and evaluation capacity; training national managers; Number of agreements/proposals: 11. Activity name: Sustainability: Civil Society/General; Description of activities: Training/seminars/workshops for implementing partners, other nongovernmental officials; general and other capacity building and sustainability activities where the level of government is not specified; Number of agreements/proposals: 4. Source: GAO analysis of NGO agreements and WFP proposals and country programs from fiscal years 2008 through 2010. Note: Table 10 is not meant to be a comprehensive list of complementary activities in NGOs' agreements and WFP's proposals and country programs. We categorized the list of activities into education, nutrition, and sustainability-related activities, based on our analysis of each activity's relevance to each objective. The implementing partners also agreed to perform a range of activities that either addressed multiple objectives, or addressed other objectives. For these activities that could not be categorized under these three objectives or that may have fallen under more than one objective, we omitted them from the table. [End of table] [End of section] Appendix VII: MGD Program's Performance Indicators in NGOs Agreements and WFP Proposals: USDA has developed over 30 performance indicators for the MGD Program. These indicators are based on the program's objectives of education, nutrition, and sustainability. We analyzed all 19 NGOs agreements and all 12 WFP proposals and country programs from fiscal years 2008 through 2010 to identify the indicators on which each implementing partner agreed to report. Table 11 shows a full list of the indicators implementing partners agreed to report and the number of NGOs agreements and WFP proposals that include each indicator. Table 11: MGD Program's Performance Indicators in NGOs Agreements and WFP Proposals: Educational access: Performance indicators: Percentage increase in enrollment - boys; Number of agreements/proposals: 28. Performance indicators: Percentage increase in enrollment - girls; Number of agreements/proposals: 31. Performance indicators: Percentage increase in attendance - boys; Number of agreements/proposals: 27. Performance indicators: Percentage increase in attendance - girls; Number of agreements/proposals: 29. Performance indicators: Percentage increase in attendance (gender not specified); Number of agreements/proposals: 0. Performance indicators: Number of take-home rations distributed; Number of agreements/proposals: 0. Performance indicators: Number of food supplements provided; Number of agreements/proposals: 0. Performance indicators: Other; Number of agreements/proposals: 1. Educational progress: Performance indicators: Percentage of boys and girls entering grade 1 of primary school who reach grade 3 (or grade 5) multiyear agreements (cohort survival to grades 3 or 5); Number of agreements/proposals: 24. Performance indicators: Promotion rate; Number of agreements/proposals: 26. Performance indicators: Continuation rate; Number of agreements/proposals: 24. Performance indicators: Percentage of targeted schools with adequate school supplies; Number of agreements/proposals: 21. Performance indicators: Percentage of targeted schools with printed materials to support literacy and numeric instruction; Number of agreements/proposals: 15. Performance indicators: Percentage/number of teachers receiving training; Number of agreements/proposals: 23. Performance indicators: Number of communities with adult literacy classes; Number of agreements/proposals: 1. Performance indicators: Number of supply kits provided; Number of agreements/proposals: 0. Performance indicators: Other; Number of agreements/proposals: 2. Nutrition: Performance indicators: Number of daily meals provided; Number of agreements/proposals: 27. Performance indicators: Number of take-home rations distributed; Number of agreements/proposals: 17. Performance indicators: Percentage of participating schools implementing health and nutrition education for students; Number of agreements/proposals: 24. Performance indicators: Percentage of participating schools with established prophylactic programs; Number of agreements/proposals: 21. Performance indicators: Percentage of participating schools with established health care provider to the schools including preschools; Number of agreements/proposals: 7. Performance indicators: Percentage of mother and child clinics/health facilities supported; Number of agreements/proposals: 2. Performance indicators: Percentage of children on target with age-to- weight and height growth; Number of agreements/proposals: 5. Performance indicators: Percentage of schools that institutionalized health and hygiene programs; Number of agreements/proposals: 1. Performance indicators: Other; Number of agreements/proposals: 0. Sustainability: Performance indicators: Number of other donors contributing to ancillary projects of complementary activities (school infrastructure, immunization program); Number of agreements/proposals: 26. Performance indicators: Dollar amount of donor contributions; Number of agreements/proposals: 15. Performance indicators: Percentage of cost sharing; Number of agreements/proposals: 16. Performance indicators: Percentage of donor support; Number of agreements/proposals: 13. Performance indicators: Implementing partner/participant's/ organization's independent contribution; Number of agreements/proposals: 14. Performance indicators: Percentage of schools where parents provide a defined level of support; Number of agreements/proposals: 27. Performance indicators: Percentage of schools with parent groups (associations or councils) that play a defined role in program management; Number of agreements/proposals: 30. Performance indicators: Number of complementary programs; Number of agreements/proposals: 29. Performance indicators: Number of potable water projects; Number of agreements/proposals: 24. Performance indicators: Number of latrine and/or kitchen repairs and constructions; Number of agreements/proposals: 25. Performance indicators: Number of nutrition and health education classes for adults; Number of agreements/proposals: 21. Performance indicators: Percentage of schools transitioning into parent-teacher association/government-supported feeding program; Number of agreements/proposals: 19. Performance indicators: Percentage of government increased support to education sector; Number of agreements/proposals: 25. Performance indicators: Number of training sessions for capacity building; Number of agreements/proposals: 28. Performance indicators: Percentage of donor support vs. of indigenous support; Number of agreements/proposals: 16. Performance indicators: Number of provincial Ministries of Education that received training (in subjects like community mobilization, administration, and monitoring); Number of agreements/proposals: 0. Performance indicators: Other; Number of agreements/proposals: 0. Source: GAO analysis of NGO agreements and WFP proposals from fiscal years 2008 through 2010. Note: The MGD Program provides its prospective implementing partners with a list of more than 30 possible indicators they can use to track performance. For this analysis, we recorded the indicators that the implementing partners had selected in the NGO agreements and WFP proposals for fiscal years 2008 through 2010. Some of the indicators have zeros because no implementing partners selected them during that time period. [End of table] [End of section] Appendix VIII: Comments from the U.S. Department of Agriculture: Note: GAO comments supplementing those in the report text appear at the end of this appendix. USDA: United States Department of Agriculture: Farm and Foreign Agricultural Service: Foreign Agricultural Service: 1400 Independence Ave, SW: Washington, DC 20250-1034: May 4, 2011: Mr. Thomas Melito: Director, International Affairs and Trade Team: United States Government Accountability Office: 441 G Street, N.W. Washington, D.C. 20548: Dear Mr. Melito: The U.S. Department of Agriculture (USDA) appreciates this opportunity to review the Government Accountability Office (GAO) draft report on "International School Feeding: USDA's Oversight of the McGovern-Dole Food for Education Program Needs Improvement" (GAO-11-544). USDA is pleased that the GAO noted our positive results in the grant approval and agreement negotiation processes, financial oversight, and performance indicators. The Department also notes the GAO's recommendations that oversight of the McGovern-Dole program continue to improve, and that internal controls he strengthened to ensure that program funds arc used as intended. USDA agrees with these recommendations and will work to address them. USDA takes program management very seriously and recognizes the essential role that the proper monitoring and evaluation of projects plays in providing Oversight of current programs, improving the quality of Entire programs, and measuring results. USDA has taken several actions over the past four years to make improvements. These actions include new hires, increasing the number of site visits by three-fold, requiring mid-term and final evaluations of projects. incorporating results-oriented management (ROM) procedures, and improving technology to better capture and analyze performance data. USDA appreciates the GAO's acknowledgment of the steps USDA already has taken to improve program management. Establishing an Improved Monitoring Process: The GAO recommends that USDA establish a systematic monitoring process that includes analyzing and reporting on a pre-selected set of indicators that directly measure the program's progress towards its objectives. During the past two years USDA has hired additional staff to review project reports and conduct site visits. As a result, USDA has increased the number of site visits substantially and has established a target of assessing each project in the field at least once every two years. USDA also has developed a field monitoring guide for staff in order to standardize the results of field monitoring trips. USDA will institute a systematic process for choosing which projects to visit and will finalize its monitoring guidance. USDA includes a set of pre-selected indicators in its McGovern-Dole program agreements but is currently working through a ROM process to improve the measurement of results for learning and health. USDA will identify a common set of indicators on which all program participants will be required to report. These common indicators will permit more effective program management, an assessment of program impact. and allows USDA to compare performance across programs. USDA also will require performance monitoring plans front program participants. All this will he implemented in the FY2012 program cycle. Finally, in March USDA launched its new Food Aid Information System (FAIS), an integrated online application through which USDA manages its food assistance programs and coordinates with program participants. Future program participants will he required to submit all required reports in FAIS. The new FAIS allows USDA to more easily compare and contrast different elements of McGovern-Dole program reporting to facilitate discussions regarding project challenges. opportunities. and corrective actions. Prioritizing Evaluations of Projects: USDA is committed to building a sustainable culture of program accountability and transparency that fosters learning from experience. To support this the Department changed its regulations to require mid- term and final evaluations on protects. USDA also has drafted a monitoring and evaluation policy. That policy outlines the purpose of monitoring and evaluation. the range of methods used to monitor and evaluate programs (including experimental and quasi-experimental methods, roles and responsibilities of USDA and program stakeholders). and the ways in which evaluations will he used and disseminated to inform decisions regarding program management and implementation. This policy will be released by September 2011. USDA currently reviews evaluation reports received from McGovern-Dole program participants during the closeout process. However, the evaluation policies and procedures currently being developed will apply to all implementing partners. All evaluation reports will be monitored and reviewed by USDA. Formalizing Procedures for Closing Out Grants: USDA agrees that closing out agreements in a timely manner is important to ensure full accountability in all projects. USDA expects to close most agreements within 180 days of the receipt of final reports and documents. The closeout process does take longer in the limited number of cases where it latent issue with an agreement, or a return of funds is required. USDA has improved its tracking of the closeout process to continue to improve on timeliness. [See comment 1] USDA also has set a goal or closing 65 percent of 2003 and 2004 agreements his September 30, 2011, and is on track to meet this goal. In addition, the Monitoring and Evaluation Staff continues to close more recent agreements as they are submitted and ready for closure to ensure there is no backlog of agreements. USDA will take another look at its closeout procedures to seek additional improvements including establishing timeframes for organizations to submit required documents, and holding those that don't meet deadlines accountable. USDA notes that it has never had a backlog of McGovern-Dole agreements awaiting closeout. USDA thanks GAO for its review of the McGovern-Dole program. All comments and recommendations will he taken into account and USDA will work to finalize all needed policies and procedures. Sincerely. Signed by: John Brewer: Administrator: Foreign Agricultural Service: The following is GAO's comment on the USDA's letter dated May 4, 2011. GAO Comments: 1. USDA stated that it has never had a backlog of McGovern-Dole Program agreements awaiting closeout; however, we found that 15 of 42 (36 percent) of MGD Program grants awarded before 2008 have completed program activities but have not yet been closed by USDA. USDA acknowledged the importance of closing out grant agreements in a timely manner in order to ensure full accountability in projects, and stated that it expects to close out agreements within 180 days of the receipt of the final reports and documents. [End of section] Appendix IX: GAO Contact and Staff Acknowledgments: GAO Contact: Thomas Melito, (202) 512-9601 or melitot@gao.gov: Staff Acknowledgments: In addition to the individual named above, Phillip Thomas, Assistant Director; Barbara Shields, Analyst-in-Charge; Sada Aksartova; Ming Chen; Debbie Chung; Martin De Alteriis; Marissa Dondoe; Tanya Doriss; Mark Dowling; Daniel Egan; Etana Finkler; Fang He; Glenn Slocum, and Adam Vogt made key contributions to this report. Other contributors include Kenneth Bulle, Lynn Cothern, Karen Deans, Julie Hirshen, Joy Labez, Jeremy Latimer, Armetha Liles, Taylor Nelson, Julia Roberts, Cristina Ruggiero, Ellery Scott, Cynthia Taylor, Celia Thomas, and Melissa Zoock. [End of section] Related GAO Products: International Food Assistance: Better Nutrition and Quality Control Can Further Improve U.S. Food Aid. [hyperlink, http://www.gao.gov/products/GAO-11-491]. Washington, D.C.: May 12, 2011. Global Food Security: U.S. Agencies Progressing on Governmentwide Strategy, but Approach Faces Several Vulnerabilities. [hyperlink, http://www.gao.gov/products/GAO-10-352]. Washington, D.C.: March 11, 2010. International Food Assistance: A U.S. Governmentwide Strategy Could Accelerate Progress toward Global Food Security. [hyperlink, http://www.gao.gov/products/GAO-10-212T]. Washington, D.C.: October 29, 2009. International Food Assistance: Key Issues for Congressional Oversight. [hyperlink, http://www.gao.gov/products/GAO-09-977SP]. Washington, D.C.: September 30, 2009. International Food Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts. [hyperlink, http://www.gao.gov/products/GAO-09-980]. Washington, D.C.: September 28, 2009. International Food Assistance: Local and Regional Procurement Provides Opportunities to Enhance U.S. Food Aid, but Challenges May Constrain Its Implementation. [hyperlink, http://www.gao.gov/products/GAO-09-757T]. Washington, D.C.: June 4, 2009. International Food Assistance: Local and Regional Procurement Can Enhance the Efficiency of U.S. Food Aid, but Challenges May Constrain Its Implementation. [hyperlink, http://www.gao.gov/products/GAO-09-570]. Washington, D.C.: May 29, 2009. USAID Acquisition and Assistance: Challenges Remain in Developing and Implementing a Strategic Workforce Plan. [hyperlink, http://www.gao.gov/products/GAO-09-607T]. Washington, D.C.: April 28, 2009. Foreign Aid Reform: Comprehensive Strategy, Interagency Coordination, and Operational Improvements Would Bolster Current Efforts. [hyperlink, http://www.gao.gov/products/GAO-09-192]. Washington, D.C.: April 17, 2009. Foreign Assistance: State Department Foreign Aid Information Systems Have Improved Change Management Practices but Do Not Follow Risk Management Best Practices. [hyperlink, http://www.gao.gov/products/GAO-09-52R]. Washington, D.C.: November 21, 2008. USAID Acquisition and Assistance: Actions Needed to Develop and Implement a Strategic Workforce Plan. [hyperlink, http://www.gao.gov/products/GAO-08-1059]. Washington, D.C.: September 26, 2008. International Food Security: Insufficient Efforts by Host Governments and Donors Threaten Progress to Halve Hunger in Sub-Saharan Africa by 2015. [hyperlink, http://www.gao.gov/products/GAO-08-680]. Washington, D.C.: May 29, 2008. Somalia: Several Challenges Limit U.S. International Stabilization, Humanitarian, and Development Efforts. [hyperlink, http://www.gao.gov/products/GAO-08-351]. Washington, D.C.: February 19, 2008. Foreign Assistance: Various Challenges Limit the Efficiency and Effectiveness of U.S. Food Aid. [hyperlink, http://www.gao.gov/products/GAO-07-905T]. Washington, D.C.: May 24, 2007. Foreign Assistance: Various Challenges Impede the Efficiency and Effectiveness of U.S. Food Aid. [hyperlink, http://www.gao.gov/products/GAO-07-560]. Washington, D.C.: April 13, 2007. Foreign Assistance: U.S. Agencies Face Challenges to Improving the Efficiency and Effectiveness of Food Aid. [hyperlink, http://www.gao.gov/products/GAO-07-616T]. Washington, D.C.: March 21, 2007. Darfur Crisis: Progress in Aid and Peace Monitoring Threatened by Ongoing Violence and Operational Challenges. [hyperlink, http://www.gao.gov/products/GAO-07-9]. Washington, D.C.: November 9, 2006. Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies. [hyperlink, http://www.gao.gov/products/GAO-06-15]. Washington, D.C.: October 21, 2005. Maritime Security Fleet: Many Factors Determine Impact of Potential Limits of Food Aid Shipments. [hyperlink, http://www.gao.gov/products/GAO-04-1065]. Washington, D.C.: September 13, 2004. United Nations: Observations on the Oil for Food Program and Iraq's Food Security. [hyperlink, http://www.gao.gov/products/GAO-04-880T]. Washington, D.C.: June 16, 2004. Foreign Assistance: Lack of Strategic Focus and Obstacles to Agricultural Recovery Threaten Afghanistan's Stability. [hyperlink, http://www.gao.gov/products/GAO-03-607]. Washington, D.C.: June 30, 2003. Foreign Assistance: Sustained Efforts Needed to Help Southern Africa Recover from Food Crisis. [hyperlink, http://www.gao.gov/products/GAO-03-644]. Washington, D.C.: June 25, 2003. Food Aid: Experience of U.S. Programs Suggest Opportunities for Improvement. [hyperlink, http://www.gao.gov/products/GAO-02-801T]. Washington, D.C.: June 4, 2002. Foreign Assistance: Global Food for Education Initiative Faces Challenges for Successful Implementation. [hyperlink, http://www.gao.gov/products/GAO-02-328]. Washington, D.C.: February 28, 2002. Foreign Assistance: U.S. Food Aid Program to Russia Had Weak Internal Controls. [hyperlink, http://www.gao.gov/products/GAO/NSIAD/AIMD-00-329]. Washington, D.C.: September 29, 2000. Foreign Assistance: U.S. Bilateral Food Assistance to North Korea Had Mixed Results. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-175]. Washington, D.C.: June 15, 2000. Managing for Results: Barriers to Interagency Coordination. [hyperlink, http://www.gao.gov/products/GAO/GGD-00-106]. Washington, D.C.: March 29, 2000. Foreign Assistance: Donation of U.S. Planting Seed to Russia in 1999 Had Weaknesses. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-91]. Washington, D.C.: March 9, 2000. Foreign Assistance: North Korea Restricts Food Aid Monitoring. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-35]. Washington, D.C.: October 8, 1999. Food Security: Factors That Could Affect Progress Toward Meeting World Food Summit Goals. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-99-15]. Washington, D.C.: March 22, 1999. Food Security: Preparations for the 1996 World Food Summit. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-97-44]. Washington, D.C.: November 7, 1996. [End of section] Footnotes: [1] WFP is a part of the United Nations system and is the world's largest humanitarian agency addressing hunger worldwide. [2] GAO, Foreign Assistance: Global Food for Education Initiative Faces Challenges for Successful Implementation, [hyperlink, http://www.gao.gov/products/GAO-02-328] (Washington, D.C.: Feb. 28, 2002). [3] This work includes a recently issued GAO report International Food Assistance: Better Nutrition and Quality Control Can Further Improve U.S. Food Aid, [hyperlink, http://www.gao.gov/products/GAO-11-491] (Washington, D.C.: May 12, 2011) and an ongoing review of the monetization of U.S. food aid. See page 13 of ]hyperlink, http://www.gao.gov/Products/Gao-11-491] for an interactive graphic with video illustrating the 3-stage supply chain required to deliver U.S. food aid from vendor to village. [4] The Millennium Development Goals are eight international development goals that United Nations member states agreed in 2000 to achieve by 2015. The eight goals are to (1) eradicate extreme poverty and hunger; (2) achieve universal primary education; (3) promote gender equality and empower women; (4) reduce child mortality rate; (5) improve maternal health; (6) combat HIV/AIDS, malaria, and other diseases; (7) ensure environmental sustainability; and (8) develop a global partnership for development. [5] The commodities includes bulgur, vegetable oil, dry beans, corn soy blend, rice, canned salmon, cornmeal, flour, wheat, wheat soy blend, and soybean meal. [6] CCC is a funding mechanism for U.S. farm income support and disaster assistance programs. Its activities, including acquisition, storage, and disposition of surplus commodities, are carried out primarily by personnel of USDA's Farm Service Agency. [7] USDA defines an intermediate result as any result below the highest level result in the MGD results framework that contributes to the achievement of the highest level result. [8] USDA defines its education objective as: (1) access, entry, and continuation, which includes enrollment and attendance rates; and (2) educational progress, which includes student achievement levels through improvements in the learning environment, such as teacher training, materials and books, etc. [9] USDA defines nutrition as the improvement in the food security and nutritional status of the target population. [10] The international donor community and experts have been placing a stronger emphasis on early childhood nutrition and maternal health. USDA has also initiated a micronutrient pilot as part of the MGD Program to test new products with a potential to improve the health and nutrition of school-age children, preschoolers, and mothers. [11] USDA defines sustainability/graduation as activities that involve host country, community, and other donor contributions that enable the project to continue after MGD Program funding ceases. [12] USDA regulations allow the agency discretion to exclude United Nations agencies from relevant reporting requirements. See 7 CFR 1599.13, 7 CFR 3019.2(cc). Based on a framework agreement between FAS and WFP signed in June 2010, beginning in fiscal year 2012, WFP is required to comply with reporting requirements similar to those of the NGOs. [13] See 7 U.S.C. 1736o-1(j). [14] FAS initiated a total of 19 agreements with the NGOs and 12 with the WFP during fiscal years 2008 through 2010. To analyze comparable information for the WFP and the NGOs, we also reviewed the WFP's proposals and country programs for the same period. These agreements represent total funding of about $202 million for all years of the NGOs' programs and $170 million for the WFP's program. [15] The programs in Pakistan planned to provide take-home rations to girls and did not have an in-school feeding component. [16] WFP indicated that it feeds preschoolers in some additional countries even if it has not specified in the agreements and proposals. [17] In fiscal year 2006, USDA began funding multiyear agreements that enable most of the MGD Program-supported projects to operate for up to 3 years, without reapplying for funding, providing more time in which implementing partners can conduct activities intended to advance the projects' sustainability. [18] In this case, performance indicators refers to outputs, such as number of beneficiaries served, number of meals served, etc., and to outcomes, such as the changes in enrollment and attendance that are reported by the implementing partners. [19] USAID may not have related programs in all the MGD countries; therefore, coordination with USAID may not be possible for all projects. One implementing partner indicated that it may not report all of its coordination with USAID in the documents we reviewed. [20] USAID's budget for maternal and child health and nutrition was about $549 million in fiscal year 2010. USAID also spent about $7 million on food for education development programs in 5 countries, including 2 MGD Program countries, in fiscal year 2010. [21] In May 2010, the administration announced a global food security initiative, Feed the Future, which is designed to provide a framework for greater interagency coordination. [22] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [23] These progress reports include logistics and monetization reports submitted by the NGOs and the standard project reports submitted by WFP but do not include the project status reports submitted by the NGOs, which are discussed below. These reports are also reviewed by USDA to verify that the NGO spent the funds as approved in the NGO's agreement with USDA. See appendix III for a description of the reports implementing partners are required to submit to FAS. We analyzed 8 NGO and 6 WFP reports beginning in 2004. [24] Education includes educational access and progress. According to USDA, indicators for educational access include attendance and enrollment. Indicators for educational progress include the number of school supplies, the number of teacher trainings, and promotion rates. [25] Based on a framework agreement between FAS and WFP signed in June 2010, beginning in fiscal year 2012, WFP is required to comply with reporting requirements similar to those of the NGOs. See appendix III for detailed information on implementing partners' reporting requirements. [26] Based on USDA's categorization, and to be consistent with our analysis of NGOs progress reporting, we included output indicators for nutrition, such as the number of take-home rations and the number of children receiving deworming treatment. [27] [hyperlink, http://www.gao.gov/products/GAO-03-143]. [28] We examined the 19 agreements that FAS initiated with NGOs and 12 that it initiated with WFP during fiscal years 2008 through 2010. For our analysis of indicators, we reviewed the NGOs' agreements and WFP's proposals. [29] Agricultural attachés and counselors are USDA foreign agricultural service officers stationed in foreign countries to provide direction and oversight to all FAS programs and to provide support and counsel to other USDA agencies. Their duties include: commodity analysis and reporting; strategic results and market access; agricultural trade, market promotion, and development; supervision and management; and mission support, coordination, and representation. [30] Mozambique's USDA FAS staff consists of a locally-hired agricultural specialist rather than an attaché. [31] WFP funds school feeding in emergency settings of protracted relief and recovery operations. [32] According to AEA, "such policies and procedures should identify the kinds of evaluations to be performed and the criteria and administrative steps for developing evaluation plans and setting priorities, selecting evaluation approaches and methods to use, consulting subject matter experts, ensuring evaluation product quality, publishing evaluation reports, ensuring independence of the evaluation function, using an appropriate mix of staff and outside consultants and contractors, appropriately focusing evaluation designs and contracts, and promoting the professional development of evaluation staff." See AEA, An Evaluation Roadmap for a More Effective Government, 2010. [33] USDA officials currently estimate that the required evaluations will cost about $50,000 per award. [34] According to USDA, single-year MGD Program awards do not require an interim evaluation. [35] This annual conference, jointly sponsored by USDA and USAID, is the largest food aid and agricultural development conference in the United States with participants representing the food and agriculture industries, maritime and rail transportation, ports, NGOs, as well as U.S. and foreign governments. [36] WFP defines impact evaluation as an assessment of whether a program or an intervention has contributed to lasting and/or significant change (positive and/or negative) for individuals, gender and age-groups, households, communities and institutions. See WFP, Impact Evaluations Concept Note 2010-2011. [37] For a discussion of evaluation planning at federal agencies with mature evaluation capacity, see [hyperlink, http://www.gao.gov/products/GAO-11-176]. [38] In the past USDA provided funding for evaluations that were proposed by NGOs as part of their MGD Program-supported activities. For example, USDA funded third-party assessments of Project Concern International's MGD project in Bolivia in 2006 and Catholic Relief Services' MGD project in Benin in 2009. While these assessments concluded that the NGOs were achieving the MGD Program's goals, they varied in their methodological rigor. Whereas the Bolivia study relied exclusively on focus groups and interviews with a range of stakeholders, the Benin study complemented qualitative findings with analyses of quantitative data on a number of indicators, such as attendance and enrollment. [39] Compliance is responsible for: (1) protecting FAS resources against fraud, waste, and abuse; (2) verifying compliance with all laws, regulations, and policies controlling FAS programs and administration; (3) promoting maximum economy and effectiveness throughout FAS; and (4) administering FAS's Security and Emergency Preparedness Programs. [40] The Food for Progress program, authorized by the Food for Progress Act of 1985, provides for the donation or credit sale of U.S. commodities to developing countries and emerging democracies to support democracy and the expansion of private enterprise. The commodities donated through Food for Progress may be sold in the recipient country and the proceeds may be used to support agricultural, economic, or infrastructure development programs. [41] Monetization is the practice of selling commodities to generate cash resources for development projects. Some MGD Program grant agreements allow the NGO to sell USDA-provided commodities to generate revenue for program implementation. [42] Financial compliance reviews are separate from programmatic monitoring visits and therefore are not counted in the visits discussed in the earlier section on monitoring. [43] The Single Audit Act requires states, local governments, and nonprofit organizations expending $500,000 or more in federal awards in a year to obtain an audit in accordance with the requirements set forth in the act. A Single Audit consists of (1) an audit and opinions on the fair presentation of the financial statements and the Schedule of Expenditures of Federal Awards; (2) gaining an understanding of and testing internal control over financial reporting and the entity's compliance with laws, regulations, and contract or grant provisions that have a direct and material effect on certain federal programs (i.e., the program requirements); and (3) an audit and an opinion on compliance with applicable program requirements for certain federal programs. For a description of these reports, see appendix III. [44] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [45] See appendix III for a description of the reports that implementing partners are required to submit to USDA. As noted previously, WFP does not submit the same reports to USDA as the NGOs. Thus, the closeout procedure for WFP is different. USDA closes out a WFP agreement when all budgeted funds are spent or when WFP notifies USDA that no further payments under the agreement are required. [46] GAO, Grants Management: Attention Needed to Address Undisbursed Balances in Expired Grant Accounts, [hyperlink, http://www.gao.gov/products/GAO-08-432] (Washington, D.C.: Aug. 29, 2008). [47] Foreign Agricultural Service Private Voluntary Organization Grant Fund Accountability: no. 07016-1-At (issued March 2006) and Foreign Agricultural Service Implementation of the Trade Title of the 2002 Farm Bill and the 2002 President's Management Agenda: no. 50601-12-At (issued March 2007). [48] In response to the OIG's reports, USDA engaged a public accounting firm to conduct a review of FAS's internal controls. The firm's report was released in July 2009 and corroborated the OIG's findings. [49] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: Nov. 1999). [50] GAO, Performance Measurement and Evaluation: Definitions and Relationships, [hyperlink, http://www.gao.gov/products/GAO-05-739SP] (Washington, D.C.: May, 2005). GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143], (Washington, D.C.: Nov. 22, 2002). GAO, Program Evaluation: Experienced Agencies Follow a Similar Model for Prioritizing Research, [hyperlink, http://www.gao.gov/products/GAO-11-176] (Washington, D.C.: Jan. 2011). GAO, International Food Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts, [hyperlink, http://www.gao.gov/products/GAO-09-980] (Washington, D.C.: Sept. 28, 2009). [51] We combined the NGO agreements for Liberia in fiscal years 2009 and 2010 because we determined that the 2010 agreement represented a modification of the prior agreement. [52] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [53] Foreign Agricultural Service Private Voluntary Organization Grant Fund Accountability: no. 07016-1-At (issued March 2006) and Foreign Agricultural Service Implementation of the Trade Title of the 2002 Farm Bill and the 2002 President's Management Agenda: no. 50601-12-At (issued March 2007). [54] These studies evaluated programs focused on primary school aged children in developing countries in field settings and examined the effect of these programs on education and/or health and/or sustainability outcomes. In addition, these studies also focused on programs that provided either in-school meals or take-home rations. Studies only assessing the impact of micronutrient fortification were not included. [55] The summary of selected studies on school feeding was conducted by one economist at GAO and then independently reviewed by two other economists at GAO who concurred with the analysis presented here. [56] The total amount appropriated for the MGD Program ($931 million) includes amounts funded through the Commodity Credit Corporation since 2003. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: