K-12 Education: Education Needs to Improve Oversight of Its 21st Century Program
Highlights
What GAO Found
The Department of Education (Education) awards 21st Century Community Learning Centers (21st Century) grants to states, who in turn, competitively award funds to local organizations, which use them to offer academic enrichment and other activities to improve students' academic and behavioral outcomes. In their most recent grant competitions, states awarded 21st Century funds to nearly 2,400 organizations—including school districts and community-based organizations—based on a variety of criteria, such as the quality of their proposed program designs. Relevant research we reviewed that compared program participants to those of non-participants suggests that the 21st Century program is effective in improving students' behavioral outcomes, such as school-day attendance and reduced disciplinary incidents, more often than their academic outcomes. However, because Education's current 21st Century performance measures primarily focus on students' reading and math scores on state tests, Education lacks useful data about whether the program is achieving its objectives to improve students' behavioral outcomes such as attendance and discipline—the areas where the program most frequently has a positive effect. Education officials have not substantially revised the program's performance measures since 1998, in part because its authorization lapsed from fiscal years 2008 through 2016. Leading practices in performance measurement call for federal agencies to align performance measures with program objectives.
21st Century Community Learning Centers' Objectives and Performance Measures for Student Outcomes
Education's technical assistance to states does not adequately address challenges states face in evaluating their 21st Century programs and sustaining them when program funding ends. About a third of states reported in GAO's 50-state survey that they face challenges in evaluating program performance, such as difficulty designing evaluations that shed light on program effects. Further, the 21st Century program was reauthorized twice, which resulted in significant changes to state requirements for evaluating programs. However, Education has not provided states written guidance on developing and conducting high-quality evaluations since 1999. Federal standards for internal control state that when significant changes occur agencies should periodically review policies for continued relevance and effectiveness in achieving their objectives. Absent written guidance to states on conducting high-quality evaluations, Education may miss opportunities to help states improve their capacity to conduct such evaluations to assure the program is meeting its goals.
Why GAO Did This Study
Education's 21st Century program—funded about $1 billion annually since 2002—supports a broad array of activities outside the school day to improve student outcomes in high-poverty or low-performing K-12 schools. A statement accompanying the Consolidated and Further Continuing Appropriations Act of 2015 included a provision for GAO to review Education programs outside the regular school day.
GAO examined (1) how 21st Century funds are awarded and used, (2) what is known about the effectiveness of these programs, (3) how Education manages and uses program data to inform decision making, and (4) Education's technical assistance for evaluating and sustaining programs. GAO conducted a 50-state survey of program officials, obtaining a 100 percent response rate. GAO also reviewed selected state program evaluations and academic studies on student outcomes, and observed program activities and interviewed officials in four states representing a range of grant size and location.
Recommendations
GAO is making four recommendations, including that Education expand its performance measures for behavioral outcomes and provide written guidance to states on conducting high-quality program evaluations. Education neither agreed nor disagreed with the recommendations, and outlined steps it is taking to address them.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Education | The Secretary of Education should direct the Office of Academic Improvement to expand its performance measures for the 21st Century program to address all program objectives. Specifically, Education should establish performance measures related to key behavioral, including student attendance and disciplinary incidents, and socio-emotional outcomes. |
In 2020, Education revised the program's performance measures to address all program objectives, including improvements in student achievement, attendance, and key behavioral outcomes. For example, Education now requires states to collect baseline data on the percentage of regular program participants: 1) whose mathematics grades improve from fall to spring; 2) who improve from not proficient to proficient or above on State assessments, and; 3) whose teachers report improvements in student behavior.
|
| Department of Education | The Secretary of Education should direct the Office of Academic Improvement to conduct federal-level data checks on the accuracy of 21st Century program data submitted by states. Such checks could test for logical relationships between fields. Education should also publicly disclose and address any data limitations it identifies, as appropriate. |
Education has implemented system changes to ensure the accuracy of 21st Century program data submitted by states. In September 2017, Education implemented system data checks on the existing 21st Century program data system. This included checks for: 1) logical relationships between data fields to identify and eliminate data errors; and 2) incomplete data fields with alerts for state educational agencies (SEA) if data is not entered in the correct fields. In addition, Education added a function for printable records for authorized users. This provides the SEA with a significant second option for conducting validation and accuracy checks. Furthermore, Education assigned extended learning time provider (ELT) designation to the SEA to ensure data is reported correctly because some grantees previously reported erroneously that they were ELT providers. Finally, in September 2017, Education updated the system to reject entries when the system does not recognize zip codes entered by grantees/sub-grantees, such as zip codes that are not found under facilities in the National Centers for Education Statistics database.
|
| Department of Education | The Secretary of Education should direct the Office of Academic Improvement to provide written, non-regulatory guidance to states on developing and conducting high-quality 21st Century state evaluations to help address the difficulties states face in measuring program performance and effectiveness. |
Education has taken several steps to implement this recommendation. Specifically, Education has provided written, non-regulatory guidance and peer-to-peer technical assistance to states on developing and conducting high-quality statewide evaluations. This included issuing additional guidance in measuring program performance and effectiveness, selecting experienced program evaluators, and selecting the best evidence-based evaluation strategies. For example, in mid- to late 2017, Education conducted three webinars for state coordinators on strategies and approaches for statewide evaluations. In April 2018, Education provided state coordinators with a webinar focused on conducting effective program evaluations to help address the difficulties states face in measuring program performance and effectiveness. Finally, in January 2019, Education provided state coordinators a webinar focused on best practices and processes for selecting program evaluators. Education made these presentations available in written format to be accessed at later dates.
|
| Department of Education | The Secretary of Education should direct the Office of Academic Improvement to use the information it collects from its monitoring visits and ongoing interactions with states to share effective practices across states for sustaining their 21st Century programs once program funding ends. This information could be shared using existing mechanisms such as Education's meetings with 21st Century state coordinators. |
Education took several steps to implement this recommendation. Specifically, since 2019 Education has provided technical assistance and shared effective practices across states for sustaining 21st Century programs through several mechanisms, including the increased use of webinars, communities of practice, and other technology-enabled tools to support the expanded sharing of effective practices GAO recommended. For example, in January 2019, Education published the 21st Century New Directors' Toolkit, which provides strategies and practices for program monitoring, evaluation, contract development and competition management, as well as support for subgrantees. In addition, Education provided non-regulatory guidance on its website to states and sub-grantees for sustainability planning. This included guidance on developing sustainability plans and guidance for new grantees.
|