Program Evaluation: Strategies to Facilitate Agencies' Use of Evaluation in Program Management and Policy Making

What GAO Found

In a governmentwide survey, GAO found that most federal managers lack recentevaluations of their programs. Thirty-seven percent reported that an evaluation had been completed within the past 5 years of any program, operation, or project they were involved in. Another 40 percent of managers reported that they did not know if an evaluation had been completed. However, 80 percent of managers who did have evaluations reported that those evaluations contributed to a moderate or greater extent to improving program management or performance and to assessing program effectiveness or value. Fewer reported that evaluations contributed moderately or more to allocating resources within a program (67 percent) or streamlining programs (61 percent).

Of the 37 percent of federal managers who had evaluations, the factor most often rated as having hindered use to a great or very great extent was lack of resources to implement the evaluation findings (33 percent). The next most frequently reported barriers related to program context, such as resolving differences of opinion among program stakeholders (23 percent). Other issues were not considered significant barriers by these managers, such as the lack of credibility or timeliness of study results, lack of leadership commitment or support for using evaluations, or difficulty accepting unexpected findings. Managers reported limited knowledge of congressional support for using results; 39 percent reported not being able to judge whether this was a barrier.

The agency evaluators GAO interviewed noted that it usually takes a number of studies, rather than just one, to influence change in programs or policies. They described using evaluations to modify existing or develop new programs and share what works with their program partners. They emphasized three basic strategies to facilitate evaluation influence:

1. Demonstrate leadership support of evaluation for accountability and improvement by promoting capacity building and the use of evidence and funding evaluation offices to promote and support the use of evidence.

2. Build a strong body of evidence by attending to rigor in whatever methods are used and accumulating a knowledge base from which to respond to varied questions over time or fast-breaking policy discussions.

3. Engage stakeholders throughout the evaluation process--developing relationships to gain their input to planning and buy-in; providing assistance, training, and incentives; and disseminating usable messages.

GAO observes that

• Agencies' lack of evaluations may be the greatest barrier to their informing program managers and policy makers.

• Seeking out in advance the interests and concerns of program stakeholders, including Congress, can help ensure that evaluations will provide the information necessary for effective management and oversight.

• Comprehensive evaluations that examine the coverage and effectiveness of federal programs and policies aimed at achieving similar outcomes could be key to coordinating and streamlining programs to reduce duplication and overlap.

Why GAO Did This Study

The GPRA Modernization Act of 2010 (GPRAMA) aims to ensure that agencies use performance information in decision making and holds them accountable for achieving results and improving government performance. GPRAMA requires GAO to evaluate the act's implementation; this report is one of a series to assess its initial implementation. GAO examined the extent of agencies' use of program evaluations--a particular form of performance information, factors that may hinder their use in program management and policy making, and strategies that may facilitate their use.

GAO surveyed a stratified random sample of 4,391 federal civilian managers and supervisors to obtain their perspectives on several results-oriented management topics, including the extent of and barriers to their evaluation use. GAO also interviewed the Office of Management and Budget and evaluators on barriers to evaluation use and strategies to facilitate it at five agencies selected for their evaluation experience in the Departments of Agriculture, Health and Human Services, and Labor. These officials' views cannot be generalized but provide useful insights.