Workforce Investment Act: DOL Should Do More to Improve the Quality of Participant Data
Highlights
What GAO Found
Flexibility in the Department of Labor's (DOL) data reporting guidance and limitations in some state information systems continue to impair the quality of the data on participants in the Workforce Investment Act (WIA) Adult and Dislocated Worker Programs. The flexibility in the guidance stems from the inherent nature of WIA, which allows states and local areas to tailor program design and service delivery to their needs. As a result, DOL's guidance on collecting and reporting the data allows variation in how some WIA data elements are defined, collected, and reported. Specifically, an American Job Center--formerly known as a one-stop center--can choose to provide certain basic services exclusively through WIA programs, exclusively through a partner program, or through a blend of both WIA and partner programs. However, this flexibility involves variations in data reporting that have contributed to inconsistencies among states regarding when job seekers are counted as WIA participants. Moreover, some aspects of DOL's guidance are open to interpretation, leaving it to states to define variables such as type of training service received, further contributing to data inconsistencies. In addition, some state information systems used to collect and report WIA participant data have limitations that hamper the affected states' ability to report uniform and complete data. For example, data are incomplete to the extent that states may not have information systems that can track participants who access services online without significant staff assistance. Having inconsistent and incomplete data makes it difficult for DOL to compare data on program participants across states or to aggregate the data at a national level.
DOL engages in various oversight activities designed to ensure the accuracy of states' data on participants in the WIA Adult and Dislocated Worker Programs and has taken steps to improve data consistency across states. However, DOL does not consistently use the results of its oversight to identify and resolve systemic data issues nor has it evaluated the effect of oversight on the quality of WIA participant data. Specifically, DOL requires states to validate the data they collect and report on participants in the WIA Adult and Dislocated Worker Programs on an annual basis, but it does not strategically use the findings from this effort to identify systemic data issues or improve the quality of the data. Similarly, although DOL's regional offices review a sample of each state's WIA participant files every few years to assess states' compliance with data reporting and validation requirements, DOL officials said they have not analyzed the findings from the most recent reviews to identify nation-wide reporting issues. DOL has taken steps to improve the consistency of the data by providing general technical assistance to states and local areas and through standardizing the way DOL collects WIA data. For example, since 2007, two states have been piloting a unified reporting system developed by DOL that uses standardized data definitions and is integrated across certain American Job Center programs administered by DOL. However, DOL officials said they have no plans to evaluate the system before expanding it to other states. Without an evaluation, DOL does not know what impact the pilot has had on the quality of WIA participant data.
Why GAO Did This Study
Having reliable program data is important in effectively managing a program. However, there have been longstanding concerns about the quality of data on job seekers enrolled in the WIA Adult and Dislocated Worker Programs, which rely on states and local areas to track participants and the services they receive. Given these concerns and WIA's anticipated reauthorization, GAO was asked to examine the data on these WIA participants. This report addresses: (1) the factors that have affected the ability to report consistent and complete data on participants in the WIA Adult and Dislocated Worker Programs, and (2) actions that DOL has taken to improve the quality of these data. To conduct this work, GAO reviewed relevant federal laws, regulations, guidance, and documentation from DOL. GAO interviewed officials from DOL's national and regional offices and state and local workforce officials from a nongeneralizable sample of eight states. GAO also analyzed WIA data from program year 2011 to determine the number of, characteristics of, and services provided to WIA participants.
Recommendations
GAO recommends that DOL take steps to improve the consistency and completeness of data reported across states and to promote a continuous process for improving the data's quality. DOL officials did not agree or disagree with GAO's overall recommendations and detailed how data quality is being addressed primarily through existing efforts. However, GAO believes that the recommendations remain valid as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Labor | To improve the consistency and completeness of national data on participants in the WIA Adult and Dislocated Worker Programs, the Secretary of Labor should take additional steps to improve the uniformity of participant data reported by states. These steps could include the following: (a) providing additional guidance to states on data reporting, such as how core and intensive services should be recorded for WIA participants who receive these services through partner programs; and (b) conducting an evaluation or review of Workforce Investment Streamlined Performance Reporting System (WISPR) to determine if it has resulted in more complete and consistent data collection and reporting for participants in the WIA Adult and Dislocated Worker Programs and placing a high priority on the implementation of WISPR if it is shown to improve data consistency and completeness. |
In July 2014, Congress enacted the Workforce Innovation and Opportunity Act (WIOA), superseding the Workforce Investment Act. Recently, DOL aligned several WIOA and other programs into a shared record layout for participant data to help ensure data accuracy and to better facilitate comparisons across programs. In addition, the new record layout allows fewer "blank fields". Namely, by eliminating the option to leave fields blank, it increases the likelihood that data entry will be accurate and complete while decreasing the likelihood that data fields will unintentionally be left blank.
|
| Department of Labor | To improve the consistency and completeness of national data on participants in the WIA Adult and Dislocated Worker Programs, the Secretary of Labor should take additional steps to improve the uniformity of participant data reported by states. The Secretary of Labor should promote a formal, continuous process for improving the quality of data on participants in the WIA Adult and Dislocated Worker Programs through such measures as the following: (a) consistently sharing the results of all oversight activities with states and local areas, including findings from validation of participant data; (b) reviewing the methods used for data validation, such as its scope and error rate threshold, to identify opportunities to increase efficiencies and accountability in the process. This could include implementing, if appropriate, recommendations from the Regions' review of data validation procedures; (c) evaluating data validation efforts to determine their effects on data quality, particularly on systemic errors, and providing targeted guidance and assistance to states and local areas to address such errors; (d) regularly monitoring Social Policy Research Associates' corrections and analyses of state WIA participant data, sharing this information with states, and coordinating with states to ensure that any corrections are appropriate and accurate; and (e) collecting and disseminating promising practices to states and local areas on data collection and reporting on a regular basis. |
States began using a Participant Individual Record Layout (PIRL)--which includes common reporting definitions across WIOA core programs--as of July 1, 2016, and DOL anticipates receiving the first State Annual Performance Reports based on these data by October 16, 2017. As part of this effort, DOL has held in-person training sessions with state and local officials and also provided them with technical assistance, including guidance related to implementing required data collection and reporting efforts under WIOA. In addition, DOL has implemented several data validation efforts, such as adding additional edit checks and identifying, addressing technical issues as they occur, and incorporating user feedback.
|