Evidence-Based Policymaking: Agencies Need Additional Guidance to Assess Their Capacity
Fast Facts
Federal decision-makers need evidence—such as data and the results of studies—to determine if programs are working as intended and to identify potential improvements.
As part of a government-wide push to improve the collection and use of evidence, agencies are required to assess their related efforts every 4 years.
Of the 23 agencies we reviewed, 21 said the assessment process helped them to better understand what they were doing to collect and use evidence, and their capacity to do so. But 18 agencies also had trouble interpreting guidance and doing their assessments.
Our recommendations are to help improve the assessment process.
Highlights
What GAO Found
The Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act) requires agencies, every 4 years, to assess aspects of their statistics, evaluation, research, and analysis efforts by addressing five topics, such as quality and methods. Agencies published capacity assessments for the first time in 2022.
GAO identified 12 themes related to six topics across agencies' assessments.
Agency officials identified common benefits and challenges related to developing capacity assessments. Officials expect some challenges will not persist. For example, officials stated it was challenging to identify all evidence-building activities because they are dispersed around their agencies, but they have now developed a better understanding of where those activities occurred.
Agency officials also identified unresolved challenges:
- Guidance. Agency officials said they faced challenges understanding Office of Management and Budget (OMB) guidance and how the assessments would be used. GAO found variation in what agencies assessed and how they presented their findings. Additional guidance could help agencies provide more comparable information in future iterations. This could also help decision-makers identify and address common issues across agencies.
- Methods. Agency officials said it was a challenge to identify appropriate approaches for conducting the assessment. They used different methodologies to assess their evidence-building capacity, which sometimes did not result in useful information. Identifying, documenting, and sharing lessons learned could help agencies select appropriate methodologies to ensure future capacity assessments consistently provide useful information.
The interagency Evaluation Officer Council, chaired by OMB, has responsibilities for sharing information and helping agencies with Evidence Act implementation.
Why GAO Did This Study
Federal decision-makers need evidence about whether federal programs and activities are achieving intended results. The Evidence Act aims to enhance federal agencies' capacity to build and use evidence.
The Evidence Act includes provisions for GAO to report on findings and trends in agencies' capacity assessments. This report describes (1) common themes in agencies' capacity assessments, and (2) benefits and challenges related to conducting capacity assessments identified by agency officials.
To address these objectives, GAO conducted a content analysis of 23 agencies' capacity assessments. GAO also interviewed OMB and Evaluation Officer Council staff as well as officials at the 24 agencies directed by OMB to conduct capacity assessments—those covered by the Chief Financial Officers Act of 1990.
Recommendations
GAO recommends that OMB should leverage the Evaluation Officer Council to (1) identify agency officials' needs for additional guidance on capacity assessments and address them accordingly and (2) identify, document, and share lessons learned on capacity assessment methods. OMB neither agreed nor disagreed with the recommendations and stated that it would take them into consideration moving forward. The Departments of Agriculture and the Treasury, Nuclear Regulatory Commission, Social Security Administration, and U.S. Agency for International Development also provided comments. The remaining agencies did not comment.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | The Director of OMB should leverage the Evaluation Officer Council to identify agency officials' needs for additional guidance on capacity assessments and address them accordingly. (Recommendation 1) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Office of Management and Budget | The Director of OMB should leverage the Evaluation Officer Council to identify, document, and share lessons learned from agency officials on capacity assessment methods. This could be done periodically to ensure any new lessons are captured. (Recommendation 2) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|