Child And Family Services Reviews:
States and HHS Face Challenges in Assessing and Improving State Performance
GAO-04-781T, May 13, 2004
In 2001, the Department of Health and Human Services' (HHS) Administration for Children and Families (ACF) implemented the Child and Family Services Reviews (CFSR) to increase states' accountability. The CFSR uses states' data profiles and statewide assessments, as well as interviews and an on-site case review, to measure state performance on 14 outcomes and systemic factors, including child well-being and the provision of caseworker training. The CFSR also requires progress on a program improvement plan (PIP); otherwise ACF may apply financial penalties. This testimony is based on our April 2004 report and addresses (1) ACF's and the states' experiences preparing for and conducting the statewide assessments and on-site reviews; (2) ACF's and the states' experiences developing, funding, and implementing items in PIPs; and (3) any additional efforts that ACF has taken beyond the CFSR to improve state performance. For the April 2004 report, we surveyed all 50 states, the District of Columbia, and Puerto Rico regarding their experiences throughout the CFSR process, visited 5 states to obtain first-hand information, and conducted a content analysis of all 31 available PIPs as of January 1, 2004. We also interviewed HHS officials--including those in all 10 regional offices--and key child welfare experts.
ACF and many state officials perceive the CFSR as a valuable process and a substantial undertaking, but some data enhancements could improve its reliability. ACF staff in 8 of the 10 regions considered the CFSR a helpful tool to improve outcomes for children. Further, 26 of 36 states responding to a relevant question in our survey commented that they generally or completely agreed with the results of the final CFSR report, even though none of the 41 states with final CFSR reports released through 2003 has achieved substantial conformity on all 14 outcomes and systemic factors. Additionally, both ACF and the states have dedicated substantial financial and staff resources to the process. Nevertheless, several state officials and child welfare experts we interviewed questioned the accuracy of the data used in the review process. While ACF officials contend that stakeholder interviews and case reviews complement the data profiles, many state officials and experts reported that additional data from the statewide assessment could bolster the evaluation of state performance. Program improvement planning is under way, but uncertainties have affected the development, funding, and implementation of state PIPs. Officials from 3 of the 5 states we visited said ACF's PIP-related instructions were unclear, and at least 9 states reported in our survey that challenges to implementing their plans include insufficient funding, staff, and time. While ACF has provided some guidance, ACF and state officials remain uncertain about PIP monitoring efforts and how ACF will apply financial penalties if states fail to achieve their stated PIP objectives. Since 2001, ACF's focus has been almost exclusively on the CFSRs and regional staff report limitations in providing assistance to states in helping them to meet key federal goals. While staff from half of ACF's regions told us they would like to provide more targeted assistance to states, and state officials in all 5 of the states we visited said that ACF's existing technical assistance efforts could be improved, ACF officials acknowledged that regional staff might still be adjusting to the new way ACF oversees child welfare programs. In the April 2004 report, we recommended that the Secretary of HHS ensure that ACF uses the best available data to measure state performance. We also recommended that the Secretary clarify PIP guidance and provide guidance to regional officials on how to better integrate their many oversight responsibilities. In commenting on a draft of the April 2004 report, HHS acknowledged that the CFSR is a new process that continues to evolve, and noted several steps it has taken to address the data quality concerns we raised in that report.