Head Start:

A More Comprehensive Risk Management Strategy and Data Improvements Could Further Strengthen Program Oversight

GAO-08-221: Published: Feb 12, 2008. Publicly Released: Feb 12, 2008.

Additional Materials:

Contact:

Kay E. Brown
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In February 2005, GAO issued a report that raised concerns about the effectiveness of the Department of Health and Human Services (HHS) Administration for Children and Families' (ACF) oversight of about 1,600 local organizations that receive nearly $7 billion in Head Start grants. GAO was asked to report on (1) ACF's progress in conducting a risk assessment of the Head Start program and ensuring the accuracy and reliability of data from its annual Program Information Report (PIR) survey of grantees, (2) efforts to improve on-site monitoring of grantees, and (3) how data are used to improve oversight and help grantees meet program standards. For this report, GAO surveyed a nationally representative sample of Head Start program directors and interviewed ACF officials. GAO also reviewed ACF studies on the validity of PIR data and conducted tests of data from the 2006 PIR database.

ACF has not undertaken a comprehensive assessment of risks that may limit Head Start's ability to meet federal program objectives, despite GAO's 2005 recommendation, and little progress has been made to ensure that the data from its annual PIR survey of grantees, which could facilitate such an assessment, are reliable. To conduct a comprehensive risk assessment, ACF needs to identify external and internal risks, estimate their significance, and decide how to best manage them. While ACF says it is working to establish two systems to address programwide risk, our analysis suggests that these systems fall short of that goal. The first system, by which ACF assesses grantees before providing new funds each year, only assesses risk posed to the program by poorly performing grantees and does not allow for a broader assessment of other sources of risk, such as improper payments to contractors. The second system, a new, integrated management information system, has been in development for over 4 years and it is not clear how it will facilitate a more comprehensive risk assessment for the Head Start program. Both initiatives depend, in part, on data from the annual PIR survey of grantees, which have been found to be unreliable. ACF has taken steps to improve oversight of Head Start grantees by implementing a more rigorous process for certifying reviewers who conduct on-site monitoring visits, implementing new processes to improve the consistency of reviews, and working to establish a system for evaluating reviews on an ongoing basis. Now, ACF verifies reviewers' qualifications and requires them to pass online tests in writing and computer literacy. Reviewers must also complete ongoing training and are evaluated by their peers at the end of each review. ACF has also taken a number of steps to improve the consistency and objectivity of reviews, including developing a Web-based data collection tool to facilitate information gathering, assigning review team leaders from outside the grantee's home region to increase independence, and centralizing the review and preparation of monitoring reports. ACF is also working to establish an ongoing system for evaluating its on-site review process. ACF uses data to track grantee performance and target assistance to underperforming grantees, but weaknesses may have hindered these efforts to improve grantee performance. For example, ACF does not have clear criteria for determining which grantees need additional oversight as part of its refunding analysis. Such decisions are made on an ad-hoc basis, which may result in grantees with similar problems receiving different levels of oversight. Moreover, prior to the December 2007 reauthorization of Head Start, ACF was limited in its ability to increase competition for grants to replace underperforming grantees. Under the new law, ACF will have more flexibility to open competition for Head Start grants to other prospective grantees when current grantees fail to deliver high-quality, comprehensive Head Start programs.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Office of Head Start (OHS) made considerable effort in the 2008-2009 PIR cycle to streamline and simplify questions. Additionally, OHS has migrated PIR data to a software system which it deemed more appropriate (Oracle to Access). OHS has implemented procedures to ensure the accuracy of the Program Information Report (PIR) system, such as instructions and definitions provided at the top of each page and questions tailored by type of program. Additionally, an import format has been defined to enable information maintained in third party vendor systems to be imported and a set of validation rules constrain and cross-check data entries programmatically. In 2012, HHS reported that the PIR error validation study for the 2010-2011 program year detailed the set of verification procedures for key PIR variables.

    Recommendation: To improve its management and oversight of the Head Start program, HHS's Assistant Secretary for Children and Families should take additional steps to ensure the accuracy of PIR data by determining which elements of the PIR are essential for program management and focus resources on a streamlined version of the PIR that would be required annually of all grantees, with the responses verified periodically. If additional information is needed to produce national estimates of a wider range of Head Start program services, ACF should include the relevant, additional data items in an expanded version of the PIR, which could be administered to a random, representative sample of grantees each year.

    Agency Affected: Department of Health and Human Services: Administration for Children and Families

  2. Status: Closed - Implemented

    Comments: HHS has reported that the fiscal portion of the Office of Head Start's monitoring protocol has been refined to conduct more detailed transaction testing in areas of known risk. HHS reports that areas tested include non-federal share, compensation, contracts, and facilities and that transactions in these areas are examined to assess whether there are unallowable costs, conflicts of interest, less than arms-length agreements, or abnormalities in procurement. According to HHS, any findings identified are reviewed for potential disallowances or possible referral to the Office of the Inspector General.

    Recommendation: To improve its management and oversight of the Head Start program, HHS's Assistant Secretary for Children and Families should look for cost-effective ways to expand ACF's efforts to comply with the Improper Payments Information Act of 2002 and to address our 2005 recommendation by collecting data on and estimating the extent of improper payments made for unallowable activities and other unauthorized purposes. These should take into account various aspects of the program and should not be limited to improper payments to grantees that enroll too many children from families that do not meet the program's income eligibility requirement.

    Agency Affected: Department of Health and Human Services: Administration for Children and Families

  3. Status: Closed - Implemented

    Comments: Office of Head Start (OHS) has created a webcast on the Head Start risk assessment process. Additionally, it has created an integrated data management system, the Head Start Enterprise System (HSES). However, the agency has not reported to what extent, if any, this system is being used to proactively manage the Head Start program nationally and assess program-wide risks. Recent GAO findings in GAO-10-604 illustrate that Head Start still lacks a comprehensive approach to centrally assess Head Start program-wide risks. HHS reported in FY12 that the HSES's capacity to track actions, trigger workflows, and generate status reports allowed OHS to manage risk and performance, For example, OHS cited use of the system to track and manage requests for waiver of preschool teacher qualifications and testing of plans to geo-map community demographics and program data that might affect the availability or demand for Head Start services.

    Recommendation: To improve its management and oversight of the Head Start program, HHS's Assistant Secretary for Children and Families should more fully implement our 2005 recommendation by developing a strategic and comprehensive approach to assessing Head Start programwide risks. A comprehensive, programwide risk assessment should take all program risks into account. Specifically, a comprehensive risk assessment should include an assessment of risks arising from external sources, such as social and demographic changes that may affect the availability or demand for Head Start program services, as well as from internal sources, such as underperforming grantees, differences in how regional offices implement program policies and procedures, or the availability of sound data to help manage the program. A comprehensive risk assessment should also include strategies for minimizing risks that could significantly limit the ability of ACF and grantees to help grantees deliver high-quality programs.

    Agency Affected: Department of Health and Human Services: Administration for Children and Families

  4. Status: Closed - Implemented

    Comments: Office of Head Start (OHS) maintains that its "Head Start Risk Management process" allows the office to prevent or reduce risk, focus on early intervention and build on strengths, improve communication and information sharing, and create corporate knowledge. OHS reports that the Risk Management Meeting process was introduced to grantees and fully implemented in 2008. It states that individual action plans are determined and the results of these meetings are entered into the Head Start Enterprise System. In FY12, OHS reported that the assessments and conditions leading to targeted reviews included: (1) assessment of data available before the site review, such as program information reports, tracking reports, and fiscal data, to determine the depth of review, (2) assessment of the fiscal information form by a federal fiscal specialist and follow-up by an on-site fiscal reviewer; (3) immediate follow-up of ad hoc reports of serious health and safety issues and periodic assessment of other complaints; (4) anniversary of first award, with new grantees automatically receiving a targeted assessment one year after award; (5) presence of serious risks, as noted in monitoring reports, regional office observations, and other sources, such as OIG reports; and (6) monitoring results that suggest potential misuse of federal funds. While it is difficult to determine from OHS' description of many of these assessments (or conditions) whether they incorporate (or constitute) sharply defined criteria, at least one -- anniversary of first award -- is a clear and objective criterion for triggering more thorough reviews.

    Recommendation: To improve its management and oversight of the Head Start program, HHS's Assistant Secretary for Children and Families should develop clear criteria for determining which grantees require more thorough reviews--such as special, on-site reviews--as a result of its refunding analysis system.

    Agency Affected: Department of Health and Human Services: Administration for Children and Families

 

Explore the full database of GAO's Open Recommendations »

Nov 13, 2014

Sep 24, 2014

Sep 10, 2014

Aug 22, 2014

Jul 16, 2014

Jun 16, 2014

Jun 4, 2014

Apr 30, 2014

Mar 12, 2014

Looking for more? Browse all our products here