USDA Is Monitoring Market Development Programs as Required but Could Improve Analysis of Impact
GAO-13-740: Published: Jul 31, 2013. Publicly Released: Aug 30, 2013.
What GAO Found
Market development program participants use program funds to support a variety of activities intended to raise awareness or acceptance of U.S. agricultural products in overseas markets. Common activities include, among others, market research, consumer and retail promotion, and participation in international trade shows. GAO's analysis of expenditure data from 2007 through 2011 shows that participants in the Market Access Program (MAP) and the Foreign Market Development Program (FMD)--the largest of the five market development programs--remained generally consistent during that period. The program participants with the largest shares of funding and the countries where the largest shares of funds were spent also remained relatively consistent. Expenditure data for 2011 show that MAP and FMD participants met or exceeded FAS contribution requirements that they match minimum percentages of the program funding they receive. Unlike funding for the other programs, a portion of MAP funds is used for promotion of branded products. In 2011, MAP participants spent about 85 percent of program funding on overseas promotion of generic commodities. More than 600 small companies and seven agricultural cooperatives spent the remaining 15 percent of MAP funding to promote branded products.
The U.S. Department of Agriculture's (USDA) Foreign Agricultural Service (FAS) uses several management and monitoring processes to reduce the risk of duplication among the five programs. FAS uses an integrated system to process funding applications for multiple programs and to monitor expenditures, which reduces the risk of duplication. According to FAS officials, FAS also monitors participants' expenses for all programs through its compliance review process. In addition, FAS guidance requires program participants to submit annual progress reports on the results of their market development activities. GAO found that performance measures in a sample of progress reports generally reflected selected FAS guidance and key attributes of successful performance measures that GAO had identified. However, the sampled reports did not always outline the methodologies used to assess activity results as required by FAS guidelines. In these cases, it would be difficult for FAS to determine the reliability of the reported results and the impact of market development activities.
A 2007 cost-benefit analysis of MAP and FMD, commissioned by FAS, found that the programs increased U.S. agricultural exports and benefited the U.S. economy, but methodological limitations may affect the magnitude of the estimated benefits. Overall, the analysis asserted that the government's expenditures for the two programs resulted in greater increases in U.S. agricultural exports and greater benefit to the U.S. economy than would have occurred without the expenditures. However, an economic model used to estimate the programs' impact on U.S. market share omitted important variables, such as commodity prices. Also, the study did not include sensitivity analyses of certain key assumptions underlying its estimates of impacts on U.S. exports. For example, analyses of the possible effects of varying levels of program funding would provide a clearer picture of the potential impact of increased or decreased funding on U.S. exports and the economy. FAS officials reported that they plan to commission a new cost-benefit analysis in 2014 but have not yet identified the methodologies that the new analysis will use.
Why GAO Did This Study
USDA administers five programs to assist U.S. agricultural industry efforts to build, maintain, and expand overseas markets. However, members of Congress continue to debate the level of funding for this assistance and its impact on agricultural exports. USDA provides about $250 million annually for the five market development programs. MAP and FMD received about 90 percent of this funding in fiscal year 2012, with allocations of $200 million and $34.5 million, respectively.
GAO was asked to review USDA's market development programs. This report (1) describes participation and expenditures in these market development programs, particularly MAP and FMD; (2) examines FAS's management and monitoring of its market development programs; and (3) assesses FAS's cost-benefit analysis of MAP's and FMD's impact on the U.S. economy. GAO analyzed USDA expenditure data from 2002 through 2011 and reviewed key agency and program participant documents. GAO also assessed a sample of participants' annual progress reports and assessed economic cost-benefit analyses of MAP and FMD commissioned by USDA.
What GAO Recommends
GAO recommends that USDA (1) emphasize that market development program participants' annual progress reports should identify the methodologies used to assess results and (2) ensure that any economic models used in future cost-benefit analyses of the programs include industry-specific variables and sensitivity analyses of key assumptions. USDA concurred with GAO's recommendations.
Recommendations for Executive Action
Comments: In its July 2013 written comments on the draft report, the U.S. Department of Agriculture (USDA) concurred with the recommendation, acknowledging the importance that every program participant's performance measures should include clear and strong methodologies. In a September 2015 email, USDA told GAO that its Foreign Agricultural Service (FAS) emphasized the importance of this element to program participants at their annual meetings held in November 2013 and 2014. In addition, beginning with the 2015 program year, FAS added the six attributes of good performance measures that GAO identified to the program application review process, making it explicit that a constraint/performance measure cannot be considered adequate without an identified methodology. USDA told GAO that FAS also informed cooperators that the adequacy of performance measures is considered in their overall rating. Feedback letters to participants from FAS in 2013 reinforced the importance of identifying the methodologies used to assess results for each performance measure. In August 2015, USDA told GAO that FAS conducted an assessment of 2015 performance measures and found consistency with the previous GAO review and a marked improvement in performance measures having an identified methodology. GAO is currently assessing the steps the agency has taken to implement this recommendation and once GAO confirms FAS's actions, the recommendations will be closed accordingly.
Recommendation: To improve MAP and FMD participants' annual reporting of the results of their market development activities, the Secretary of Agriculture should direct FAS to use appropriate means to emphasize the importance of participants' identifying the methodologies used to assess results for each performance measure in their annual country progress reports.
Agency Affected: Department of Agriculture
Comments: In is July 2013 written comments on the draft report, the U.S. Department of Agriculture (USDA) concurred with the recommendation. In a September 2015 email, USDA told GAO that its Foreign Agricultural Service (FAS) contracted for the next program-wide cost-benefit analysis in the spring of 2015. The Request for Proposal (RFP) specifically requested potential contractors to respond to GAO recommendations. USDA told GAO that a contractor was selected and is expected to address the additional variables GAO recommended for consideration. GAO is currently assessing the steps the agency has taken to implement this recommendation. Once GAO confirms FAS's actions, the recommendation will be closed accordingly.
Recommendation: To improve the accuracy of future cost-benefit analyses of FAS's market development programs, the Secretary of Agriculture should direct FAS to ensure that any econometric model used for the cost-benefit analysis, such as the market share model, includes industry-specific variables that could have a significant role in determining the U.S. market share--for example, commodity prices, production volumes, and number of export competitors.
Agency Affected: Department of Agriculture
Comments: In its July 2013 written comments to the draft report, the U.S. Department of Agriculture (USDA) concurred with the recommendation. In a September 2015 email, USDA told GAO that the request for proposal, which was contracted by the Foreign Agricultural Service's (FAS) for the next program-wide cost-benefit analysis in the spring of 2015, included a requirement for a sensitivity analysis of key assumptions. USDA told GAO that the contractor included this in their proposal and is expected to include it in their analysis. A final report is expected in the first quarter of 2016. GAO is currently assessing the steps the agency has taken to implement this recommendation. Once GAO confirms FAS's actions, the recommendation will be closed accordingly.
Recommendation: To improve the accuracy of future cost-benefit analyses of FAS's market development programs, the Secretary of Agriculture should direct FAS to conduct a sensitivity analysis, in accordance with best practices for cost estimates, of the key assumptions that are applied in any economic models used in the cost-benefit analysis, such as the market share model and spillover effect model.
Agency Affected: Department of Agriculture