Science, Technology, Engineering, and Mathematics Education: Strategic Planning Needed to Better Manage Overlapping Programs across Multiple Agencies
Highlights
What GAO Found
In fiscal year 2010, 13 federal agencies invested over $3 billion in 209 programs designed to increase knowledge of STEM fields and attainment of STEM degrees. The number of programs within agencies ranged from 3 to 46, with the Departments of Health and Human Services and Energy and the National Science Foundation administering more than half of these programs. Almost a third of the programs had obligations of $1 million or less, while some had obligations of over $100 million. Beyond programs specifically focused on STEM education, agencies funded other broad efforts that contributed to enhancing STEM education.
Eighty-three percent of the programs GAO identified overlapped to some degree with at least 1 other program in that they offered similar services to similar target groups in similar STEM fields to achieve similar objectives. Many programs have a broad scope—serving multiple target groups with multiple services. However, even when programs overlap, the services they provide and the populations they serve may differ in meaningful ways and would therefore not necessarily be duplicative. Nonetheless, the programs are similar enough that they need to be well coordinated and guided by a robust strategic plan. Currently, though, less than half of the programs GAO surveyed indicated that they coordinated with other agencies that administer similar STEM education programs. Current efforts to inventory federal STEM education activities and develop a 5-year strategic plan present an opportunity to enhance coordination, align governmentwide efforts, and improve efficiency of limited resources by identifying opportunities for program consolidation and reducing administrative costs.
Agencies’ limited use of performance measures and evaluations may hamper their ability to assess the effectiveness of their individual programs as well as the overall STEM education effort. Specifically, program officials varied in their ability to provide reliable output measures—for example, the number of students, teachers, or institutions directly served by their program. Further, most agencies did not use outcomes measures in a way that is clearly reflected in their performance planning documents. This may hinder decision makers’ ability to assess how agencies' STEM education efforts contribute to agencywide performance goals and the overall federal STEM effort. In addition, a majority of programs did not conduct comprehensive evaluations since 2005 to assess effectiveness, and the evaluations GAO reviewed did not always align with program objectives. Finally, GAO found that completed STEM education evaluation results had not always been disseminated in a fashion that facilitated knowledge sharing between both practitioners and researchers.
Why GAO Did This Study
Science, technology, engineering, and mathematics (STEM) education programs help to enhance the nation’s global competitiveness. Many federal agencies have been involved in administering these programs. Concerns have been raised about the overall effectiveness and efficiency of STEM education programs.
GAO examined (1) the number of federal agencies and programs that provided funding for STEM education programs in fiscal year 2010; (2) the extent to which STEM education programs have similar objectives, serve similar target groups, and provide similar types of services, and, if necessary, what opportunities exist to increase coordination; and (3) the extent to which STEM education programs measured effectiveness. To answer these questions, GAO reviewed relevant federal laws, regulations, and plans; surveyed federal STEM education programs; analyzed programs’ STEM evaluations; and interviewed relevant federal officials. An electronic supplement—GAO-12-110SP—provides survey results.
Recommendations
GAO recommends that as OSTP leads the governmentwide STEM education strategic planning effort, it should work with agencies to better align their activities with a governmentwide strategy, develop a plan for sustained coordination, identify programs for potential consolidation or elimination, and assist agencies in determining how to better evaluate their programs. OSTP provided technical comments that we incorporated as appropriate. OMB had no concerns with the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Science and Technology Policy | The Director of Office of Science and Technology Policy (OSTP) should direct National Science and Technology Council (NSTC) to develop guidance for how agencies can better incorporate each agency's STEM education efforts and the goals from NSTC's 5-year STEM education strategic plan into each agency's own performance plans and reports. |
The administration named STEM education as an interim cross agency priority goal in the President's 2013 budget submission and in May 2013 NSTC released its STEM Education 5-Year Strategic Plan. As a cross-agency priority goal, the administration must report quarterly on the progress agencies have made towards the goal of increasing the number of graduates with a STEM degree by 1 million over the next 10 years. In July 2013, a joint OSTP/Office of Management and Budget memorandum included guidance to agencies on how to align their programs and 2015 budget submissions with the goals of NSTC's STEM Education 5-Year Strategic Plan. This guidance could help agencies to better target their resources toward programs with positive outcomes and to align them with national STEM education goals. As of August 2013, some agencies began to align performance plans with the draft goals and priorities of NSTC's Strategic Plan and their STEM education efforts. For instance, according to OSTP, the National Science Foundation has an agency priority goal that is focused on the use of evidence-based practices in undergraduate STEM education.
|
| Office of Science and Technology Policy | The Director of OSTP should direct NSTC to develop a framework for how agencies will be monitored to ensure that they are collecting and reporting on NSTC strategic plan goals. This framework should include alternatives for a sustained focus on monitoring coordination of STEM programs if the NSTC Committee on STEM terminates in 2015 as called for in its charter. |
In May 2013, NSTC released its STEM Education 5-Year Strategic Plan, which includes roadmaps and implementation plans for STEM education investment priority areas and interagency coordination approaches. The strategic plan states that CoSTEM will create implementation subcommittees for the STEM education investment priority areas and interagency coordination approaches who will be responsible for 1) reviewing roadmaps and implementation plans, 2) tracking implementation of priority areas, 3) developing and monitoring metrics for progress, and 4) developing a framework and process for more coordinated Federal STEM education budget planning. In January 2014, NSTC extended the charter of one of the implementation subcommittees - the Federal Coordination in Science, Technology, Engineering and Mathematics Education (FC-STEM) subcommittee - until May 2018. The FC-STEM subcommittee, composed of 12 federal agencies, established an action plan that outlines subgoals linked to the 5-Year Strategic Plan and designated working groups that are responsible for quarterly reporting and monitoring progress toward these goals. The plan includes key milestones, due dates, and potential barriers to achieving the goals. Such a plan could help to enhance federal agencies' efforts to implement a coordinated STEM education approach that focuses on national priorities, as well as identify critical performance information that agencies should collect and report on that aligns with these priorities.
|
| Office of Science and Technology Policy | The Director of OSTP should direct NSTC to work with agencies, through its strategic planning process, to identify programs that might be candidates for consolidation or elimination. Specifically, this could be achieved through an analysis that includes information on program overlap, similar to the analysis conducted by GAO in this report, and information on program effectiveness. As part of this effort, OSTP should work with agency officials to identify and report any changes in statutory authority necessary to execute each specific program consolidation identified by NSTC's strategic plan. |
OSTP directed NSTC through its strategic planning process to identify programs that might be candidates for consolidation or elimination. Specifically, in the Coordinating Federal Science, Technology, Engineering, and Mathematics (STEM) Education Investments: Progress Report, published in February 2012, NSTC identified a number of programs that could be eliminated in FY13. In addition, the President's FY14 budget calls for major restructuring of federal STEM education programs through the consolidation of 90 programs and the realignment of STEM education activities.
|
| Office of Science and Technology Policy | The Director of OSTP should direct NSTC to develop guidance to help agencies determine the types of evaluations that may be feasible and appropriate for different types of STEM education programs and develop a mechanism for sharing this information across agencies. This could include guidance and sharing of information that outlines practices for evaluating similar types of programs. |
In May 2013, NSTC released its STEM Education 5-Year Strategic Plan, which includes guidance to agencies in developing evaluations for STEM education programs. Specifically, NSTC's Strategic Plan includes a coordination objective focused on improving and aligning evaluation and research strategies across federal agencies and a set of design principles that provides guidance for how federal STEM programs should be structured and evaluated, based on current evidence and best practices. In addition to NSTC's Strategic Plan, agencies have already begun to develop shared strategies for generating evidence. For instance, the National Science Foundation and the Department of Education developed a common evidence framework and guidelines for various education research study types.
|