Evidence-Based Policymaking: Selected Agencies Coordinate Activities, but Could Enhance Collaboration
Fast Facts
Agencies can bring multiple sources of information together to determine whether federal programs are working as intended — a practice known as “evidence-building.” Agencies assess existing evidence, determine whether new evidence is needed, and set priorities to get decision makers the evidence they need.
Collaboration within an agency can help ensure that evidence-building efforts are effective. While the agencies we reviewed have processes to coordinate evidence-building, we recommended that the Corporation for National and Community Service and the Departments of Labor and Health and Human Services collaborate better on setting priorities.
People with documents around a conference table
Highlights
What GAO Found
Federal decision makers need evidence about whether federal programs and activities achieve intended results as they set priorities and consider how to make progress toward national objectives. The five agencies GAO reviewed took actions that align with direction from Congress and the Office of Management and Budget (OMB) to strengthen their evidence-building activities. The five agencies are: the Departments of Education, Health and Human Services (HHS), and Labor (DOL); the Corporation for National and Community Service (CNCS); and the U.S. Agency for International Development. For example, based on a statutory requirement, a majority of grant funding for HHS's Maternal, Infant, and Early Childhood Home Visiting program is to be used for home visiting models with sufficient evidence of their effectiveness. Consistent with this requirement, HHS annually assesses evidence, such as the results of program evaluations, to identify effective home visiting models that grantees can implement.
Evidence-building can involve assessing existing evidence, identifying any new evidence needs, and prioritizing when to fulfill those needs. These efforts are fragmented within each of the five agencies—that is, each has multiple organizational units with responsibilities for evidence-building. For example, DOL has established separate units responsible for different sources of evidence—evaluations, performance information, and statistics. Effective collaboration can help agencies manage this fragmentation, and lead to improved results.
GAO found that to assess existing evidence, each agency established a coordinated, agency-wide process that reflects leading practices for collaboration. Those leading practices are: (1) defining a leadership model; (2) involving relevant participants ; (3) clarifying roles and responsibilities ; and (4) documenting that information in written guidance . However, agencies' processes for determining which new evidence to generate, when, and how (i.e., prioritizing new evidence) did not always reflect the leading practices (see figure).
Why GAO Did This Study
Congress and OMB have taken steps intended to strengthen federal evidence-building activities. In September 2017, a federal commission found that agencies had uneven capacity to support, or did not fully coordinate, a full range of evidence-building activities.
GAO was asked to examine the coordination of federal evidence-building activities. This report (1) describes selected agencies' actions that align with direction from Congress and OMB to strengthen evidence-building activities and (2) examines the extent to which selected agencies' processes for coordinating those activities reflect leading practices for collaboration.
To address these objectives, GAO reviewed documents and interviewed officials about federal evidence-building activities at five selected agencies. GAO selected these agencies based on the greater number of experiences they had in comparison to other agencies incorporating these activities into the design and implementation of certain programs. GAO assessed their coordination of these activities against four leading practices for collaboration identified in GAO's past work.
Recommendations
GAO is making a total of seven recommendations to DOL, CNCS, and HHS to better reflect leading collaboration practices in their evidence prioritization processes. DOL concurred, CNCS neither agreed nor disagreed, and HHS did not concur with the recommendations. CNCS and HHS stated, but did not provide information to support, that each had already taken relevant actions. GAO continues to believe the recommendations are valid, as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| AmeriCorps | The Chief Executive Officer of CNCS should develop an approach to ensure that all relevant participants are involved in the agency-wide process for prioritizing evidence needs. (Recommendation 1) |
In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendation and described planned actions to strengthen its agency-wide evidence prioritization process. In December 2020, Americorps (CNCS's new operating name as of September 2020) revised its process to involve relevant internal and external stakeholders. For example, the Director of the Office of Research and Evaluation, who previously did not have a defined role or responsibilities related to evidence prioritization, is to help lead evidence prioritization across the agency. In addition, Americorps' new process incorporates OMB's guidance, which states that agencies should consult with a variety of external stakeholders during the evidence prioritization process. Americorps' engagement of relevant internal and external stakeholders can help the agency meet the evidence needs of decisions makers within and outside the agency.
|
| AmeriCorps | The Chief Executive Officer of CNCS should define roles and responsibilities for all relevant participants involved in the agency-wide process for prioritizing evidence needs. (Recommendation 2) |
In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendation and described planned actions to strengthen its agency-wide evidence prioritization process. In December 2020, Americorps (CNCS's new operating name as of September 2020) revised its process to clearly define roles and responsibilities for key participants. For example, staff from program offices are to identify emerging evidence priorities across the agency. In addition, members of the agency's leadership team are to coordinate these priorities and integrate them into agency planning. By clearly defining roles and responsibilities, Americorps can ensure that participants in the process are aware of and agree upon who will do what, how they will organize individual and joint efforts, and how they will make decisions.
|
| AmeriCorps | The Chief Executive Officer of CNCS should revise written guidance for the agency-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 3) |
In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendation and described planned actions to strengthen its agency-wide evidence prioritization process. In December 2020, Americorps (CNCS's new operating name as of September 2020) issued guidance for a revised evidence prioritization process. The document identifies roles and responsibilities for relevant participants. For example, the guidance states that program office staff are to identify emerging evidence priorities by developing key policy questions and strategies for answering them. It further states that agency leadership is to coordinate and integrate evidence priorities into agency planning. By clearly documenting who should be involved, and how they should contribute, Americorps has greater assurance that relevant participants in its evidence prioritization process are effectively collaborating to prioritize the agency's evidence needs.
|
| Department of Health and Human Services | The Secretary of Health and Human Services should develop an approach to ensure that all relevant participants are involved in the department-wide process for prioritizing evidence needs. (Recommendation 4) |
In a November 2019 letter providing comments on our report, HHS stated that it was in the process of developing an approach for including all relevant participants in its evidence prioritization process, in line with our recommendation. Subsequently, in March 2020, HHS announced its new evidence prioritization process. As part of that process, HHS's operating and staff divisions are to consult with internal and external stakeholders when prioritizing evidence needs. This stakeholder engagement can help HHS meet the evidence needs of decision makers both within and outside the agency.
|
| Department of Health and Human Services | The Secretary of Health and Human Services should revise written guidance for the department-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 5) |
In a November 2019 letter providing comments on our report, HHS stated that it was in the process of developing an approach for including all relevant participants in its evidence prioritization process, in line with our recommendation. Subsequently, in March 2020, HHS announced a new evidence prioritization process, and its related guidance addresses our recommendation. Specifically, HHS's written guidance for that process identifies roles and responsibilities for key participants. For example, the guidance directs the heads of HHS's operating divisions to work with experts in research and evaluation to develop evidence priorities for their organizations. Furthermore, it directs HHS's Evaluation Officer, located within the Office of the Assistant Secretary for Planning and Evaluation, to coordinate these priorities across the department. By clearly documenting who should be involved, and how they should contribute, HHS has greater assurance that relevant participants are effectively collaborating to prioritize the agency's evidence needs.
|
| Department of Labor | The Secretary of Labor should develop an approach to ensure that all relevant participants are involved in the department-wide process for prioritizing evidence needs. (Recommendation 6) |
In an October 2019 letter commenting on our report, the Department of Labor (DOL) stated that in fiscal year 2020 it would update written guidance for its department-wide evidence-building activities to formalize the involvement of relevant participants. In September 2020, DOL updated its guidance to direct DOL component agencies to consult OMB guidance when prioritizing their evidence needs. OMB's guidance states that agencies should consult with a variety of internal and external stakeholders during the evidence prioritization process. This stakeholder engagement can help DOL meet the evidence needs of decision makers within and outside the department.
|
| Department of Labor | The Secretary of Labor should revise written guidance for the department-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 7) |
In an October 2019 letter commenting on our report, the Department of Labor (DOL) stated that in fiscal year 2020 it would update written guidance for its department-wide evidence-building activities to formalize the involvement of relevant participants and to identify related roles and responsibilities. In September 2020, DOL updated written guidance in line with our recommendation. For example, it directs each DOL component agency to designate an official to lead the agency's prioritization of its evidence needs. It also identifies roles and responsibilities at the department level to review component agencies' evidence priorities, to support a more coordinated and crosscutting approach to evidence-building. By clearly documenting who should be involved, and how they should contribute, DOL has greater assurance that relevant participants are effectively collaborating to prioritize the agency's evidence needs.
|