Vocational Rehabilitation:

Better Measures and Monitoring Could Improve the Performance of the VR Program

GAO-05-865: Published: Sep 23, 2005. Publicly Released: Sep 23, 2005.

Additional Materials:

Contact:

Daniel Bertoni
(202) 512-9889
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Education (Education) provides more than $2.5 billion annually to the states for a federal-state vocational rehabilitation (VR) program to help individuals with disabilities become employed. This program is among a large number of federal programs intended to assist people with disabilities. In 2003 GAO placed federal disability programs on its list of high-risk programs because many of these programs have not kept up with scientific advances and economic and social changes. GAO prepared this report under the Comptroller General's authority as part of an effort to assist policy makers in determining how federal disability programs could more effectively meet the needs of individuals with disabilities and addressed it to each committee of jurisdiction. In this report, GAO assesses the (1) extent to which state VR agencies assist individuals in achieving employment, and (2) performance measures and monitoring practices Education uses to manage this decentralized program and achieve legislative goals.

Of the more than 650,000 individuals exiting the state VR programs in fiscal year 2003, one-third (217,557) obtained a new job or maintained their existing job for at least 90 days after receiving services. Education's data showed that the remaining two-thirds exited the VR program without employment most often because the individual refused services or failed to cooperate with the VR counselor (46 percent of the time) or could not be located or contacted (24 percent). The VR program purchased more than $1.3 billion in services for all individuals who exited the program in fiscal year 2003, two-thirds of which were used to provide services to individuals exiting with employment. Employment, earnings, and the amount of purchased services received while in the VR program varied significantly by individuals' disability type and other characteristics. In addition, state VR agencies varied substantially in the employment rates they achieved, the characteristics of individuals they served, their frequency of providing certain services, and their service expenditures. Education's performance measures are not comprehensive, and its monitoring of state VR agencies has not resulted in timely feedback. Education does not comprehensively measure the performance of certain key populations, such as students transitioning from school to work, and tracks only the individuals who exit the program, not those still receiving services. In addition, Education's performance measures do not take into consideration all the variation among the state VR agencies or allow for comparisons with other workforce programs. Education's monitoring reports, which are its primary means of providing feedback to state VR agencies, are frequently late and based on data that are more than 2 years old. Consequently, state VR agencies do not receive the timely feedback needed to improve the efficiency and effectiveness of their programs. In managing the performance of the VR program, Education also does not censure poorly performing state VR agencies, reward strong performance, or take full advantage of opportunities to disseminate best practices. Education recently decided to eliminate its regional offices, which conducted most of the monitoring of state VR agencies, making the details of the future monitoring process unclear.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In a December 2005 letter to GAO, Education agreed that corrective action approaches are an essential component for effective program administration and said that the Rehabilitation Services Administration (RSA) is committed to improving its performance in this regard. Further, the agency noted that RSA's new organizational structure would enable RSA to review, monitor, and follow up with issues in a more consistent and cohesive manner because these activities would now be handled by a centralized staff. In addition, it said that RSA is considering both positive and negative reinforcements to encourage compliance and that one approach it was considering was widening access to monitoring and performance data within a state to contribute to constructive change. In August 2008, Education indicated that, as part of its monitoring protocol, it will create individual plans with state agencies to implement action steps to address issues of concern and expand areas of high performance. In May 2009, the agency said that it assessed the potential for developing incentives by which it could recognize the improved performance of VR agencies. However, it concluded that the Rehabilitation Act of 1973, as amended, and its implementing regulations do not provide a mechanism by which RSA can implement the provision of such incentives.

    Recommendation: To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should ensure that it consistently applies its existing consequences for failure to meet required performance targets or consider developing new consequences that it will be willing to apply in such situations. Further, Education should consider whether developing incentives such as recognition for successful performance would provide a more useful tool for encouraging good performance.

    Agency Affected: Department of Education

  2. Status: Closed - Implemented

    Comments: In a 2005 letter to GAO, Education said that Rehabilitation Services Administration (RSA) had established a targeted goal of issuing annual reports to state Vocational Rehabilitation (VR) agencies no later than 30 days after the end of the fiscal year. In addition, Education stated that when the Rehabilitation Services Administration (RSA) conducts an on-site monitoring visit of a state agency, RSA plans to issue the monitoring report to the state agency no later than 45 days after the completion of the on-site visit. These timelines, according to Education, constitute a commitment to significant improvement over past performance. As of August 31, 2007, RSA has completed its monitoring reviews of 17 states and issued draft monitoring reports within 30 days, within the 45-day window documented in Step 8 of the 2007-09 monitoring plan.

    Recommendation: To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should ensure that Education's new plan for the monitoring of state VR agency performance addresses issues such as the timeliness of monitoring reports and the frequency of on-site visits that will be necessary to adequately gauge performance.

    Agency Affected: Department of Education

  3. Status: Closed - Not Implemented

    Comments: In its 2005 letter to GAO, Education said that it would pursue other methods to improve its data collection system so that it can continue to improve its ability to analyze and report on the data more quickly. In July 2006, Education reported that the FY 2005 Rehabilitation Services Administration (RSA) (RSA-911 and RSA-113) data were available four months after the end of the fiscal year and were made available only after they were critically reviewed and edited using a computerized edit program that checked the validity, accuracy, and reasonableness of the data. Education also said that steps are being taken to improve the timeliness of the RSA-2 data through the edits available in RSA's management information system. Because most of RSA's fiscal staff are new, this database was not completed as quickly as the other two databases, but Education anticipates improved performance with the FY 2006 data. Further, Education said that reports based on performance data, which were created for the first time using FY 2005 data, are central to the profiles that are generated for each state. Education reported that state profiles using several datasets were produced for FY 2007, and will be used in FY 2008. The data will also be made available to outside users as part of RSA's efforts to make data more transparent to outsiders. As of May 2009, RSA has not made significant advances on this recommendation. The agency reported that the annual review reports for FY 2005 and FY 2006 can be found on http://rsamis.ed.gov and were issued within 30 days of the end of the calendar year. However, the FY 2007 data were not available until June of 2008 and the FY 2008 data were not available until April of 2009. The agency attributed these delays to problems obtaining the RSA 911 report from certain VR agencies. It attributed delays in the approval of the RSA-2 data to new and inexperienced staff, but expects to improve the timeliness of the RSA-2 data next fiscal year as the staff gain expertise.

    Recommendation: To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should take steps to continue improving the timeliness of the performance data Education collects and analyzes to ensure that data are available for timely feedback to state VR agencies as well as to comply with the GPRA reporting requirements established by the Office of Management and Budget.

    Agency Affected: Department of Education

  4. Status: Closed - Implemented

    Comments: In 2005, the Department of Education agreed with our recommendation and indicated that it was currently working to address this issue. In its 2008 Draft Strategic Performance Plan Goals, Objectives, and Measures for the Vocational Rehabilitation Services Program, the U.S. Department of Education has included performance measures related to transition-age youths. Specifically, they will measure (1) the rehabilitation rate for transition-age youths, (2) the number and percentage of transition-age youths who receive services under and Individual Education Plan and an Individual Plan for Employment (IPE), and (3) the annual increases (or decreases) in the percentage of transition-age youths who achieve competitive employment at closure.

    Recommendation: To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should reevaluate Education's performance measures to determine whether they reflect the agency's goals and values and ensure that sufficient data are collected to measure the performance of this program. As part of this evaluation, Education should consider (1) developing additional measures to evaluate performance relating to individuals who remain in the VR program as well as to certain special needs populations, such as transitioning students; (2) revising performance measures to account for additional factors such as the economy and demographics of the states' populations or adjusting performance targets for individual state VR agencies to address these issues, while also considering the costs and benefits associated with collecting additional data; and (3) continuing its work to develop performance measures in line with OMB's common measures.

    Agency Affected: Department of Education

  5. Status: Closed - Implemented

    Comments: In its December 2005 letter to GAO, Education agreed that this is a valuable function and would explore ways to improve its web posting and linkages within the rehabilitation and education communities. The agency said it would also consider using various Education grantees, such as rehabilitation, continuing education programs, and materials clearinghouses, as partners in this effort. In September 2007, Education reported significant progress developing a web-based dissemination and technical assistance resource that will, among other activities, provide broader access to a wide variety of vocational rehabilitation and independent living program resources. The web-based resource project will serve as the "doorway and card catalogue" for an extensive on-line repository to facilitate public access to the most current vocational rehabilitation and independent living program information, including demographic and performance measurement data, effective practices, program initiatives, and current issues, research, and literature. The Rehabilitation Services Administration (RSA) anticipates that a non-functioning prototype of the website design and template pages will be completed in late fall 2007. Upon completion of the prototype, RSA will view and comment on the design of the prototype pages. Subsequent to the review, the contractor will begin to add the functionality of the links and develop the working prototype that will become DATAR, at which time, RSA will conduct beta testing on the design of the system with a group of grantees and stakeholders. This feedback will be used to refine and complete the project. As of July 2009, RSA reported that a prototype had been developed and tested. The final production version of DATAR is currently in the final stages of revision. RSA projects that DATAR will be available to the public in December 2009.

    Recommendation: To improve Education's performance measures and its monitoring of state VR agencies, the Secretary of Education should develop alternative means of disseminating best practices among state VR agencies in light of the elimination of the regional offices, such as a central repository of best practices information.

    Agency Affected: Department of Education

 

Explore the full database of GAO's Open Recommendations »

Dec 11, 2014

Dec 10, 2014

Nov 18, 2014

Nov 13, 2014

Oct 10, 2014

Sep 30, 2014

Sep 22, 2014

Jul 9, 2014

May 14, 2014

Looking for more? Browse all our products here