Department of Education: Improved Dissemination and Timely Product Release Would Enhance the Usefulness of the What Works Clearinghouse
GAO-10-644
Published: Jul 23, 2010. Publicly Released: Jul 23, 2010.
Skip to Highlights
Highlights
In connection with the Omnibus Appropriations Act, 2009, GAO was required to study the What Works Clearinghouse (WWC), a federal source of evidence about effective education practices. Operating through a 5-year contract awarded by the U.S. Department of Education's Institute of Education Sciences (IES), the WWC reviews education research and disseminates its findings. GAO examined: (1) the extent to which the WWC review process meets accepted standards for research evaluation and how the WWC has responded to recommendations and criticism, (2) how WWC output and costs have changed over time and how its performance is measured, and (3) how WWC products are disseminated and how useful educators find them to be. To conduct its work, GAO reviewed WWC-related documents, analyzed cost and performance data, surveyed all states and a nationally representative sample of school districts, and interviewed IES officials, WWC contractors, researchers, and others.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Education | To consistently release WWC products in a timely manner, the Secretary of Education should direct IES to develop and implement strategies that help avoid future backlogs and ensure that IES's review and approval processes keep pace with increased contractor production. Strategies could include shifting IES resources to ensure sufficient staff time for managing the peer review process and streamlining its approval processes. |
According to Department of Education officials, in 2011, the agency reviewed and revised their policies and procedures for reviewing and approving What Works Clearinghouse products. They indicated that these revisions will ensure the process is scientifically sound, efficient, and results in a timely release of documents. For example, they have automated portions of the template used by reviewers to expedite their work. They have also expanded their pool of experienced peer reviewers and internal staff support, increasing their capacity to complete reviews, and provided product release schedules that demonstrate improved timeliness.
|
| Department of Education | To better track the costs and usefulness of the WWC, the Secretary of Education should direct IES to incorporate findings from its cost studies to develop performance measures related to costs, such as identifying a range of acceptable costs per product and using that information to monitor contractor spending. |
Education concurred, and noted it had already taken some steps that address the recommendation previously, and has now fully implemented this recommendation. According to Education officials, in 2010, Education completed cost studies on WWC products. The agency shared additional findings from these studies with GAO in 2011, including cost ranges established for WWC products. Education officials confirmed that these findings were being used in the evaluation criteria for the upcoming WWC contract bidding process in 2012, and that IES plans to evaluate the next contractor based on these findings.
|
| Department of Education | To better track the costs and usefulness of the WWC, the Secretary of Education should direct IES to develop performance measures related to product usefulness and periodically assess whether WWC products are meeting the needs of target audiences by gathering information on product usefulness in the proposed survey or through other means. |
Department of Education officials reported that it uses a wide variety of tools to gather consumer input, such as a Help Desk and online voting for future report topics. GAO noted that this relies on existing users and reflects the views of those users who provide feedback, rather than those of the broader population. According to Education officials, the agency developed a plan for obtaining feedback on What Works Clearinghouse (WWC) products from attendees at major educational conferences. The agency e-mailed all conference participants and provided them with a link to their web-based customer satisfaction survey, which they shared with GAO. The agency provided GAO with some of their survey data. Education plans to conduct a more extensive review in 2012 that requires OMB clearance, including surveying potential customers about satisfaction, relevance and usefulness of WWC products. Education officials confirmed that data on product usefulness were being used in the evaluation criteria for the upcoming WWC contract bidding process in 2012, and that ED's Institute of Education Sciences will evaluate the next contractor based on these findings.
|
| Department of Education | To reach more members of the target audience, the Secretary of Education should direct IES to assess and improve its dissemination efforts to promote greater awareness and use of the WWC, for example, by developing a way to inform school districts of new products or encouraging educator professional development programs to focus on research-based practices such as those discussed in practice guides. |
According to Department of Education officials, in 2010, Education updated its outreach plan for the What Works Clearinghouse (WWC). For example, Education held department-wide briefings on WWC products intended to strengthen collaboration and awareness within the agency and improve promotion of WWC efforts. The agency also increased outreach efforts to key external educational associations in an attempt to further improve dissemination efforts. Education has also initiated several efforts to assess its dissemination efforts, including tracking of WWC product awareness and use at outreach events focused on Practice Guides and analyzing product website visits and referrals from other websites.
|
Full Report
Topics
Cost analysisDatabasesEducation program evaluationEducational researchFinancial managementGovernment information disseminationInformation disclosureInformation managementInformation systemsMonitoringPerformance measuresReporting requirementsResearch and developmentResearch programsResearch reportsStandardsStrategic planningSurveysPolicies and proceduresElectronic mail