Vocational Rehabilitation: Better Measures and Monitoring Could Improve the Performance of the VR Program
GAO-05-865
Published: Sep 23, 2005. Publicly Released: Sep 23, 2005.
Skip to Highlights
Highlights
The Department of Education (Education) provides more than $2.5 billion annually to the states for a federal-state vocational rehabilitation (VR) program to help individuals with disabilities become employed. This program is among a large number of federal programs intended to assist people with disabilities. In 2003 GAO placed federal disability programs on its list of high-risk programs because many of these programs have not kept up with scientific advances and economic and social changes. GAO prepared this report under the Comptroller General's authority as part of an effort to assist policy makers in determining how federal disability programs could more effectively meet the needs of individuals with disabilities and addressed it to each committee of jurisdiction. In this report, GAO assesses the (1) extent to which state VR agencies assist individuals in achieving employment, and (2) performance measures and monitoring practices Education uses to manage this decentralized program and achieve legislative goals.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Education | To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should reevaluate Education's performance measures to determine whether they reflect the agency's goals and values and ensure that sufficient data are collected to measure the performance of this program. As part of this evaluation, Education should consider (1) developing additional measures to evaluate performance relating to individuals who remain in the VR program as well as to certain special needs populations, such as transitioning students; (2) revising performance measures to account for additional factors such as the economy and demographics of the states' populations or adjusting performance targets for individual state VR agencies to address these issues, while also considering the costs and benefits associated with collecting additional data; and (3) continuing its work to develop performance measures in line with OMB's common measures. |
In 2005, the Department of Education agreed with our recommendation and indicated that it was currently working to address this issue. In its 2008 Draft Strategic Performance Plan Goals, Objectives, and Measures for the Vocational Rehabilitation Services Program, the U.S. Department of Education has included performance measures related to transition-age youths. Specifically, they will measure (1) the rehabilitation rate for transition-age youths, (2) the number and percentage of transition-age youths who receive services under and Individual Education Plan and an Individual Plan for Employment (IPE), and (3) the annual increases (or decreases) in the percentage of transition-age youths who achieve competitive employment at closure.
|
| Department of Education | To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should take steps to continue improving the timeliness of the performance data Education collects and analyzes to ensure that data are available for timely feedback to state VR agencies as well as to comply with the GPRA reporting requirements established by the Office of Management and Budget. |
In its 2005 letter to GAO, Education said that it would pursue other methods to improve its data collection system so that it can continue to improve its ability to analyze and report on the data more quickly. In July 2006, Education reported that the FY 2005 Rehabilitation Services Administration (RSA) (RSA-911 and RSA-113) data were available four months after the end of the fiscal year and were made available only after they were critically reviewed and edited using a computerized edit program that checked the validity, accuracy, and reasonableness of the data. Education also said that steps are being taken to improve the timeliness of the RSA-2 data through the edits available in RSA's management information system. Because most of RSA's fiscal staff are new, this database was not completed as quickly as the other two databases, but Education anticipates improved performance with the FY 2006 data. Further, Education said that reports based on performance data, which were created for the first time using FY 2005 data, are central to the profiles that are generated for each state. Education reported that state profiles using several datasets were produced for FY 2007, and will be used in FY 2008. The data will also be made available to outside users as part of RSA's efforts to make data more transparent to outsiders. As of May 2009, RSA has not made significant advances on this recommendation. The agency reported that the annual review reports for FY 2005 and FY 2006 can be found on http://rsamis.ed.gov and were issued within 30 days of the end of the calendar year. However, the FY 2007 data were not available until June of 2008 and the FY 2008 data were not available until April of 2009. The agency attributed these delays to problems obtaining the RSA 911 report from certain VR agencies. It attributed delays in the approval of the RSA-2 data to new and inexperienced staff, but expects to improve the timeliness of the RSA-2 data next fiscal year as the staff gain expertise.
|
| Department of Education | To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should ensure that Education's new plan for the monitoring of state VR agency performance addresses issues such as the timeliness of monitoring reports and the frequency of on-site visits that will be necessary to adequately gauge performance. |
In a 2005 letter to GAO, Education said that Rehabilitation Services Administration (RSA) had established a targeted goal of issuing annual reports to state Vocational Rehabilitation (VR) agencies no later than 30 days after the end of the fiscal year. In addition, Education stated that when the Rehabilitation Services Administration (RSA) conducts an on-site monitoring visit of a state agency, RSA plans to issue the monitoring report to the state agency no later than 45 days after the completion of the on-site visit. These timelines, according to Education, constitute a commitment to significant improvement over past performance. As of August 31, 2007, RSA has completed its monitoring reviews of 17 states and issued draft monitoring reports within 30 days, within the 45-day window documented in Step 8 of the 2007-09 monitoring plan.
|
| Department of Education | To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should ensure that it consistently applies its existing consequences for failure to meet required performance targets or consider developing new consequences that it will be willing to apply in such situations. Further, Education should consider whether developing incentives such as recognition for successful performance would provide a more useful tool for encouraging good performance. |
In a December 2005 letter to GAO, Education agreed that corrective action approaches are an essential component for effective program administration and said that the Rehabilitation Services Administration (RSA) is committed to improving its performance in this regard. Further, the agency noted that RSA's new organizational structure would enable RSA to review, monitor, and follow up with issues in a more consistent and cohesive manner because these activities would now be handled by a centralized staff. In addition, it said that RSA is considering both positive and negative reinforcements to encourage compliance and that one approach it was considering was widening access to monitoring and performance data within a state to contribute to constructive change. In August 2008, Education indicated that, as part of its monitoring protocol, it will create individual plans with state agencies to implement action steps to address issues of concern and expand areas of high performance. In May 2009, the agency said that it assessed the potential for developing incentives by which it could recognize the improved performance of VR agencies. However, it concluded that the Rehabilitation Act of 1973, as amended, and its implementing regulations do not provide a mechanism by which RSA can implement the provision of such incentives.
|
| Department of Education | To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should develop alternative means of disseminating best practices among state VR agencies in light of the elimination of the regional offices, such as a central repository of best practices information. |
In its December 2005 letter to GAO, Education agreed that this is a valuable function and would explore ways to improve its web posting and linkages within the rehabilitation and education communities. The agency said it would also consider using various Education grantees, such as rehabilitation, continuing education programs, and materials clearinghouses, as partners in this effort. In September 2007, Education reported significant progress developing a web-based dissemination and technical assistance resource that will, among other activities, provide broader access to a wide variety of vocational rehabilitation and independent living program resources. The web-based resource project will serve as the "doorway and card catalogue" for an extensive on-line repository to facilitate public access to the most current vocational rehabilitation and independent living program information, including demographic and performance measurement data, effective practices, program initiatives, and current issues, research, and literature. The Rehabilitation Services Administration (RSA) anticipates that a non-functioning prototype of the website design and template pages will be completed in late fall 2007. Upon completion of the prototype, RSA will view and comment on the design of the prototype pages. Subsequent to the review, the contractor will begin to add the functionality of the links and develop the working prototype that will become DATAR, at which time, RSA will conduct beta testing on the design of the system with a group of grantees and stakeholders. This feedback will be used to refine and complete the project. As of July 2009, RSA reported that a prototype had been developed and tested. The final production version of DATAR is currently in the final stages of revision. RSA projects that DATAR will be available to the public in December 2009.
|
Full Report
GAO Contacts
Public Inquiries
Topics
Aid for the disabledEmployment assistance programsEmployment of the disabledFederal aid to statesstate relationsMonitoringPerformance appraisalPerformance measuresProgram evaluationProgram managementState-administered programsStrategic planningVocational rehabilitation