This is the accessible text file for GAO report number GAO-14-137 
entitled "Workforce Investment Act: Strategies Needed to Improve 
Certain Training Outcome Data" which was released on March 5, 2014. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

January 2014: 

Workforce Investment Act: 

Strategies Needed to Improve Certain Training Outcome Data: 

GAO-14-137: 

GAO Highlights: 

Highlights of GAO-14-137, a report to congressional requesters. 

Why GAO Did This Study: 

As the economy recovers, some employers continue to face difficulty 
finding qualified workers. The WIA Adult and Dislocated Worker 
Programs provide services, including job training, which aims to help 
participants acquire skills and credentials employers need. Under WIA, 
states are required to report data on training participants who obtain 
credentials and on those who enter employment related to the training 
they receive. Given that a WIA reauthorization proposal would 
establish both of these outcomes as performance measures, GAO was 
asked to examine the capacity of states to report on these outcomes. 
This report addresses: 1) the extent to which training participants 
obtained credentials and training-related employment, 2) challenges 
states face in reporting data on credentials and what steps, if any, 
they and DOL are taking to address them, and 3) challenges states face 
in reporting data on training-related employment and what steps, if 
any, they and DOL are taking to address them. GAO interviewed DOL 
officials, workforce experts, and state and local officials and 
employer organizations from a nongeneralizable sample of six states 
selected in part on the basis of geographic diversity. GAO also 
analyzed data on credential attainment and training-related employment 
for program years 2010 and 2011. 

What GAO Found: 

Of the more than two million total participants in the Workforce 
Investment Act's (WIA) Adult and Dislocated Worker Programs, about 11 
percent and 16 percent, respectively, received training in program 
year 2011, and about two-thirds of the training participants in each 
program attained a credential. Little is known, however, about how 
many participants got jobs related to their training. From program 
year 2006 through program year 2011, the percentages of training 
participants who earned a credential declined from about 74 percent to 
58 percent for the Adult Program and from about 75 percent to 63 
percent for the Dislocated Worker Program, according to data from the 
Department of Labor (DOL). Of those training participants who attained 
a credential in program year 2011, about 65 percent earned 
occupational credentials, such as a welding certificate, followed by 
lower percentages who earned occupational skill licenses and associate'
s degrees, among others. In contrast, GAO found training-related 
employment data unreliable primarily because a significant portion of 
the data was missing. 

Workforce officials in four of six selected states cited some 
obstacles in reporting data on credential attainment, and both DOL and 
states are taking steps to address challenges. Officials in four 
states GAO contacted said reporting such data can be resource-
intensive, largely because case managers must manually track this 
information from various sources, including participants, training 
providers, and third-party organizations. To improve credential 
attainment and reporting, DOL clarified which credentials should be 
reported and began measuring credential attainment through an agency-
wide goal in 2010. Officials in five states have taken similar steps, 
such as setting goals and tracking credential attainment, and 
enhancing data exchange with training providers. Officials in most of 
the six states GAO contacted noted some obstacles to obtaining such 
data. For example, officials from several states cited student privacy 
laws as a barrier in verifying credentials with training providers. 
Officials in three states told us that they ask participants to sign 
consent forms allowing training providers to give credential 
information to local officials. 

Workforce officials in most of the selected states identified even 
greater challenges reporting data on training-related employment, 
including the high degree of resources required and the subjective 
nature of determining whether employment is linked to training. DOL 
has taken only limited steps to address these challenges. To report 
such data, case managers seek participants' employment information 
from participants, employers, and wage records. Then they must piece 
it together to determine whether participants' employment is “
substantially related” to their training. Officials in most of the six 
states described this process as resource-intensive and noted that 
making such determinations are subjective since one case manager's 
interpretation of “substantially related” may differ from another's. 
Given these challenges, officials in all six states have taken some 
steps to increase access to employment information or make decisions 
less subjective. DOL has recently stressed the importance of reporting 
data on training-related employment and shared a few practices with 
states to increase reporting rates; however, it has not identified and 
disseminated strategies to address the ongoing challenges states face 
regarding resource intensiveness and subjectivity, which could improve 
the quality of such data. 

What GAO Recommends: 

GAO recommends that DOL identify and share with states strategies that 
may ease collection and improve the quality of training-related 
employment data. DOL agreed with GAO's recommendation. 

View [hyperlink, http://www.gao.gov/products/GAO-14-137]. For more 
information, contact Andrew Sherrill at (202) 512-7215 or 
sherrilla@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOL's Data Show That Most Training Participants Attained Credentials, 
but Data on Training-Related Employment Are Unreliable: 

Selected States Cited Some Obstacles to Collecting Data on Credential 
Attainment and DOL and States Have Taken Steps to Address Challenges 
and Improve These Data: 

States Face Continuing Challenges Reporting Data on Training-Related 
Employment, Despite Efforts by States and DOL to Improve Reporting: 

Conclusions: 

Recommendation for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Labor: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Comparison of Performance Measures for the WIA Adult and 
Dislocated Worker Programs and Common Measures: 

Table 2: Variation among State-Reported Percentages of Training 
Participants Who Obtained Training-Related Employment in Program Year 
2011: 

Table 3: Key DOL Initiatives Related to Credential Attainment: 

Table 4: Establishing a Performance Measure on Credential Attainment: 
Challenges, Mitigation Strategies, and Considerations: 

Table 5: Establishing a Performance Measure on Training-Related 
Employment: Challenges, Potential Mitigation Strategies, and 
Considerations: 

Table 6: Selected Site Visit States and Local Areas: 

Figures: 

Figure 1: Overall Number of Participants in the Adult and Dislocated 
Worker Programs and Those Who Received Training, Program Years 2006 
through 2011: 

Figure 2: Percentages of Participants in the Adult and Dislocated 
Worker Programs Who Received Training, Program Years 2006 through 2011: 

Figure 3: Percentage of Training Participants in the Adult and 
Dislocated Worker Programs Who Earned Credentials, Program Years 2006 
through 2011: 

Figure 4: Percentage of Credential Types Attained by Training 
Participants in Program Year 2011: 

Figure 5: General Process for Collecting Participants' Employment 
Information and Determining Whether It Relates to their Training: 

Abbreviations: 

DOL: Department of Labor: 

FERPA: Family Educational Rights and Privacy Act: 

OJT: On-the-Job Training: 

O*NET: Occupational Information Network: 

UI: Unemployment Insurance: 

WIA: Workforce Investment Act: 

WIASRD: Workforce Investment Act Standardized Record Data: 

WIB: Workforce Investment Board: 

[End of section] 

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548: 

January 31, 2014: 

The Honorable John Kline: 
Chairman: 
Committee on Education and the Workforce: 
House of Representatives: 

The Honorable Virginia Foxx: 
Chairwoman: 
Subcommittee on Higher Education and Workforce Training: 
Committee on Education and the Workforce: 
House of Representatives: 

The Honorable Joe Heck: 
House of Representatives: 

The Honorable Howard P. McKeon: 
House of Representatives: 

During this economic recovery, some jobs go unfilled because employers 
continue to face difficulty finding qualified workers. The Department 
of Labor's (DOL) Workforce Investment Act (WIA) Adult and Dislocated 
Worker Programs provide a variety of employment services, including 
job training--which aims to help participants acquire skills and 
credentials required by employers. To assess the performance of these 
programs, WIA requires that states report on outcome measures in key 
areas including employment, retention, earnings, and credential 
attainment. WIA also requires states to negotiate expected performance 
levels for each of these measures with DOL and allows DOL to hold 
states accountable for achieving their performance goals through 
financial incentives and sanctions. In 2005, DOL began allowing states 
to request waivers to replace the WIA performance measures with a set 
of common measures to assess multiple employment and training 
programs. These measures focus on employment, retention, and earnings 
but do not include credential attainment. DOL still requires states to 
report data on training participants who received credentials. In 
addition, while it is not a WIA performance measure, WIA requires 
states to report the number of training participants who obtain 
employment in their field of training.[Footnote 1] However, DOL's 
Office of Inspector General has raised questions about the quality of 
such data. Pending WIA reauthorization legislation includes these two 
training outcomes as performance measures. You requested that we 
examine the capacity of states and local areas to report accurate and 
consistent data on such outcomes. This report addresses: 

1. To what extent do WIA training participants obtain credentials and 
secure training-related employment? 

2. What challenges do states face in reporting data on credential 
attainment and what steps, if any, are they and DOL taking to address 
them? 

3. What challenges do states face in reporting data on training-
related employment and what steps, if any, are they and DOL taking to 
address them? 

To address our objectives, we reviewed relevant federal laws, 
regulations, and DOL's guidance for the Adult and Dislocated Worker 
Programs. We also interviewed DOL officials and national experts on 
workforce issues representing six organizations. We identified these 
experts by reviewing relevant literature and using an iterative 
process to solicit recommendations from DOL officials and experts. The 
results of these interviews are not generalizable, but provided 
insights about the overall challenges states face in reporting on 
these outcomes and any efforts by states to overcome them. To more 
specifically identify the challenges states face in reporting data on 
credential attainment and employment related to training, as well as 
to learn about any steps they are taking to address these challenges, 
we interviewed state and local workforce officials from a 
nongeneralizable sample of six states: Alabama, Illinois, Kansas, 
Rhode Island, Texas, and Washington. We selected the states to reflect 
a mix of those that had leading edge data practices (as identified by 
experts) and those with either incomplete data or relatively high 
rates of reporting errors (as identified by WIA data quality reports 
on credential attainment).[Footnote 2] We also factored geographic 
diversity into state selection. In each state, we interviewed state 
workforce officials and also local workforce officials from at least 
one workforce investment board (WIB),[Footnote 3] and at least one 
American Job Center.[Footnote 4] In addition, we interviewed at least 
one employer organization in each state. In selecting these entities, 
we considered states' input on organizations that could provide us 
with an employer perspective on the value of credentials and certain 
types of training for various industries. These employer organizations 
included statewide business associations, regional business 
associations, individual employers, and industry-specific 
representatives. 

To assess the reliability of the data DOL collects on credential 
attainment and training-related employment for participants in the WIA 
Adult and Dislocated Worker Programs, we analyzed data from DOL's 
Workforce Investment Act Standardized Record Data system for 2010 and 
2011, the most recent program years for which data were available. We 
reviewed existing documentation related to reporting these data, 
tested the data electronically, assessed it for factors such as 
completeness and consistency, and interviewed knowledgeable agency 
officials. We found the data on training and credential attainment to 
be sufficiently reliable for reporting overall estimates of 
participants who entered training, the minimum number of training 
participants who earned credentials, and the types of credentials they 
attained. However, we determined that the data on employment related 
to training were not reliable for the purposes of our report, as 
discussed in more detail later in this report. See appendix I for more 
details concerning the objectives, scope, and methodology used in this 
report. 

We conducted this performance audit from October 2012 to January 2014 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit work to 
obtain sufficient, appropriate evidence to provide a reasonable basis 
for our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

WIA Programs and Administrative Structure: 

The WIA Adult and Dislocated Worker Programs provide employment 
services to a wide range of participants. The Adult Program serves all 
individuals age 18 and older, and the Dislocated Worker Program serves 
individuals who have been or will be terminated or laid off from 
employment, among others.[Footnote 5] The Adult Program prioritizes 
certain services for recipients of public assistance and other low-
income individuals when program funds are limited.[Footnote 6] To 
enable individuals to participate, both programs may offer supportive 
services such as transportation, childcare, housing, and needs-related 
payments under certain circumstances.[Footnote 7] WIA requires that 
the Adult and Dislocated Worker Programs and other federally-funded 
employment and training programs provide services through one-stop 
centers--now called American Job Centers--so that jobseekers and 
employers can find assistance at a single location. 

DOL's Employment and Training Administration administers the Adult and 
Dislocated Worker Programs and oversees their implementation, which is 
carried out by states and local areas. At the state level, the WIA 
Adult and Dislocated Worker Programs are administered by state 
workforce agencies. Each state has one or more local workforce 
investment areas, each governed by a WIB.[Footnote 8] WIBs select the 
entities that will operate American Job Centers, which provide most 
WIA services, and oversee the American Job Center network.[Footnote 9] 
WIA provides substantial flexibility to states and WIBs to determine 
how services are provided. 

Employment and Training Services: 

Provided under WIA: 

WIA represented a fundamental shift from its predecessor program, the 
Job Training Partnership Act, by decreasing the focus on job training 
as the primary means to help adults and dislocated workers get a job. 
The Adult and Dislocated Worker Programs provide participants three 
types of services: 

* Core services include basic services such as job searches and labor 
market information, and may be accessed with or without staff 
assistance. 

* Intensive services include such activities as comprehensive 
assessment and case management, which require greater staff 
involvement. Intensive services are available to participants who are 
unable to obtain or retain employment after receiving at least one 
core service. 

* Training services include such activities as occupational skills or 
on-the-job training. To be eligible for training services, 
participants must: (1) be unable to obtain or retain employment after 
receiving at least one intensive service, (2) be in need of training, 
and (3) have the skills and qualifications to successfully complete 
the training program, among other requirements.[Footnote 10] 

To assess participants' skills and determine whether they need 
training, WIBs may require them to complete certain activities. We 
previously found that most WIBs required participants to complete 
skills assessments or gather information about the occupation for 
which they wanted training before entering a training program. 
[Footnote 11] 

DOL requires that participants eligible for training select approved 
training providers in consultation with case managers, but 
participants ultimately choose the training programs in which they 
participate.[Footnote 12] WIA requires that training be directly 
linked to in-demand occupations, which DOL interprets to include both 
currently-available jobs and occupations that are projected to grow in 
the future.[Footnote 13] 

Funding, Performance Measures, and Data Quality: 

In fiscal year 2013, Congress appropriated a total of about $1.9 
billion for the Adult and Dislocated Worker Programs, roughly 24 
percent less than the $2.5 billion appropriated in fiscal year 
2000.[Footnote 14] In order to ensure that they receive their full 
funding allocations, states must meet agreed upon levels of 
performance. WIA requires states to use performance measures that 
gauge program results for job seekers in the areas of employment, 
employment retention, earnings, and credential attainment.[Footnote 
15] WIA also requires measures of customer satisfaction for jobseekers 
and for employers.[Footnote 16] WIA requires states to negotiate 
performance levels for each of these measures annually with DOL 
[Footnote 17] and provides that DOL may hold states accountable for 
achieving these levels by tying them to financial sanctions and 
incentive funding.[Footnote 18] In addition, WIA requires states to 
use Unemployment Insurance (UI) wage records to track employment-
related outcomes.[Footnote 19] 

In 2005, DOL began allowing states to request waivers to replace the 
WIA performance measures with a smaller set of common measures that 
focus on employment, retention, and earnings across multiple programs 
(see table 1). These measures do not include credential attainment and 
are calculated differently than the WIA measures.[Footnote 20] 

Table 1: Comparison of Performance Measures for the WIA Adult and 
Dislocated Worker Programs and Common Measures: 

WIA program: Adult; 
WIA performance measures[A]: 
* Entered employment rate [C]; 
* Employment retention rate at 6 months; 
* Average earnings change in 6 months; 
* Entered; 
* employment and credential rate[D]; 
Common measures[B]: 
* Entered employment[C]; 
* Employment retention; 
* Average earnings. 

WIA program: Dislocated Worker; 
WIA performance measures[A]: 
* Entered employment rate[C]; 
* Employment retention rate at 6 months; 
* Earnings replacement rate at 6 months; 
* Entered employment and credential rate[D]; 
Common measures[B]: 
* Entered employment[C]; 
* Employment retention; 
* Average earnings. 

Source: U.S. Department of Labor. 

[A] While not a performance measure, states are required to report the 
number of training participants who obtained employment related to 
their training. 

[B] While they are not performance measures, states are required to 
report both the number of training participants who earned credentials 
and the number of training participants who obtained employment 
related to their training. 

[C] "Entered employment" is defined as the percentage of participants 
who got jobs by the end of the first quarter after program exit. For 
the Adult Program, this measure excludes participants who are employed 
at the time of registration. For the Dislocated Worker Program, it 
includes those employed at the time of registration. 

[D] The Department of Labor's guidance defines the "entered employment 
and credential" measure as the percentage of trained participants who 
were employed in the first quarter after exit from the program and 
received a credential by the end of the third quarter after exit. 

[End of table] 

As of December 2013, a total of 48 states and territories, plus the 
District of Columbia, had obtained this waiver and used the common 
measures for the Adult and Dislocated Worker Programs.[Footnote 21] 
Although states and territories are no longer subject to financial 
sanctions or incentive funding for the credential attainment measure, 
DOL still requires them to report the number of training participants 
who earn credentials.[Footnote 22] In general, credential attainment 
improves workers' labor market experience through higher earnings, 
greater mobility, and enhanced job security, according to DOL and 
research studies.[Footnote 23] 

We have previously raised concerns about the accuracy and 
comparability of DOL's data on credential attainment because DOL's 
guidance allowed states and local areas considerable flexibility in 
defining what constitutes a credential.[Footnote 24] In February 2002, 
we recommended that DOL more clearly define this term. Since that 
time, DOL issued guidance to clarify its definition of credential 
attainment.[Footnote 25] In addition, to verify that the data states 
report on credential attainment are accurate, DOL requires WIA 
programs to collect and retain documentation on participants' 
credential attainment.[Footnote 26] 

In addition to the performance measures, WIA requires states to report 
a wide array of data that includes whether training participants find 
employment that relates to their training.[Footnote 27] A WIA 
participant's employment is considered to be related to the training 
received if in the new job the participant uses "a substantial portion 
of the skills taught in the training."[Footnote 28] However, DOL does 
not require that states collect and retain documentation for 
employment related to training to verify the accuracy of data they 
report. In a September 2011 report, DOL's Office of Inspector General 
raised questions about the quality of DOL's data on employment related 
to training. Specifically, the Office of Inspector General found that 
these data were "incomplete and unreliable" based on its review of the 
data DOL maintains in its WIA database.[Footnote 29] The Office of 
Inspector General recommended, among other things, that DOL provide 
guidance on the best methodology for reporting such data as well as 
provide oversight to ensure that states develop or identify best 
practices for increasing the rate of training-related employment. DOL 
agreed with these recommendations. 

Federal agencies that work in partnership with states and local areas 
to administer programs such as WIA must continually balance the 
competing objectives of collecting uniform performance data and giving 
program managers flexibility to meet local needs. Our prior work has 
found that federal agencies have considered key attributes of data 
quality for performance data, including: 

* Completeness--the extent to which enough of the required data 
elements are collected from a sufficient portion of the target 
population or sample. 

* Accuracy--the extent to which the data are free from significant 
errors. 

* Consistency--the extent to which data are collected using the same 
procedures and definitions across collectors and times. 

* Ease of use--how readily intended users can access data, aided by 
clear definitions, user-friendly software, and easy-to-use access 
procedures.[Footnote 30] 

Congressional Proposals to Reauthorize WIA Programs: 

Congress is currently considering legislation to reauthorize the 
Workforce Investment Act of 1998, which has been due for 
reauthorization since the end of fiscal year 2003. The Supporting 
Knowledge and Investing in Lifelong Skills (SKILLS) Act (H.R. 803), 
passed by the House, would establish both credential attainment and 
training-related employment as performance measures. The Workforce 
Investment Act of 2013 (S. 1356), introduced in the Senate and 
reported out by the Committee on Health, Education, Labor, and 
Pensions, would establish credential attainment as a performance 
measure, but not training-related employment. 

DOL's Data Show That Most Training Participants Attained Credentials, 
but Data on Training-Related Employment Are Unreliable: 

Most Training Participants Attained Credentials: 

During the time-period from program year 2006 through program year 
2011, the total number of participants in WIA's Adult and Dislocated 
Worker Programs increased significantly.[Footnote 31] Specifically, in 
the Adult Program, the number of participants increased from about 
625,000 to about 1.25 million, and in the Dislocated Worker Program, 
the number of participants increased from about 272,000 to about 
761,000. During this same time-period, the number of participants who 
received training services also increased, but not as dramatically as 
the number of overall participants (see figure 1).[Footnote 32] For 
example, the number of Dislocated Worker participants who received 
training increased from about 76,000 in program year 2006 to about 
120,000 in program year 2011. 

Figure 1: Overall Number of Participants in the Adult and Dislocated 
Worker Programs and Those Who Received Training, Program Years 2006 
through 2011: 

[Refer to PDF for image: stacked horizontal bar graph] 

Program year: 2006; 
Did not receive training: 
Adult: 516,696; 
Dislocated Worker: 107,952; 
Received training: 
Adult: 76,500; 
Dislocated Worker: 195,600. 

Program year: 2007; 
Did not receive training: 
Adult: 646,800; 
Dislocated Worker: 114,200; 
Received training: 
Adult: 63,600; 
Dislocated Worker: 178,400. 

Program year: 2008; 
Did not receive training: 
Adult: 927,600; 
Dislocated Worker: 113,100; 
Received training: 
Adult: 58,900; 
Dislocated Worker: 305,200. 

Program year: 2009; 
Did not receive training: 
Adult: 1,031,700; 
Dislocated Worker: 156,800; 
Received training: 
Adult: 111,200; 
Dislocated Worker: 470,600. 

Program year: 2010; 
Did not receive training: 
Adult: 1,125,500; 
Dislocated Worker: 164,500; 
Received training: 
Adult: 135,800; 
Dislocated Worker: 621,400. 

Program year: 2011; 
Did not receive training: 
Adult: 1,113,600; 
Dislocated Worker: 133,600; 
Received training: 
Adult: 120,500; 
Dislocated Worker: 640,500. 

Source: DOL data. 

Note: Data are based on participants who exited the Adult and 
Dislocated Worker Programs. Program Year 2011 reflects data for the 
time-period from April 1, 2011 through March 31, 2012, not the actual 
program year, which ran from July 1, 2011 to June 30, 2012. 

[End of figure] 

Since the number of participants who received training services did 
not increase at the same rate as the number of participants who 
entered into WIA's Adult and Dislocated Worker Programs, the 
percentage of participants who received training generally declined, 
according to DOL reports (see figure 2). 

Figure 2: Percentages of Participants in the Adult and Dislocated 
Worker Programs Who Received Training, Program Years 2006 through 2011: 

[Refer to PDF for image: vertical bar graph] 

Program year: 2006; 
Adult: 17.3%; 
Dislocated Worker: 28.1%. 

Program year: 2007; 
Adult: 13.6%; 
Dislocated Worker: 26.3%. 

Program year: 2008; 
Adult: 10.9%; 
Dislocated Worker: 16.2%. 

Program year: 2009; 
Adult: 13.2%; 
Dislocated Worker: 19.1%. 

Program year: 2010; 
Adult: 12.8%; 
Dislocated Worker: 17.9%. 

Program year: 2011; 
Adult: 10.7%; 
Dislocated Worker: 15.8%. 

Source: DOL data. 

Note: Data are based on participants who exited the Adult and 
Dislocated Worker Programs. Program year 2011 reflects data for the 
time period from April 1, 2011 through March 31, 2012, not the actual 
program year, which ran from July 1, 2011 to June 30, 2012. 

[End of figure] 

There are several reasons that may have contributed to the declining 
percentage of participants who received training. Beginning in 
December 2007, the significant increase in the overall number of 
participants is likely attributed to the downturn in the economy that 
led to a dramatic rise in unemployment[Footnote 33] and the subsequent 
infusion of additional funds to WIA programs from the American 
Recovery and Reinvestment Act of 2009 (Recovery Act).[Footnote 34] The 
Recovery Act provided $500 million for grants for the Adult Program 
and $1.25 billion for grants for the Dislocated Worker Program. By 
spring of 2009, DOL began allocating these funds to states to 
supplement existing WIA funds. DOL encouraged states to use Recovery 
Act funds to increase training in an effort to help Americans acquire 
new skills and return to work.[Footnote 35] DOL officials stated that 
the increase in demand for training services exceeded the increase in 
supply provided for by the Recovery Act, which may have led to 
shortages in training capacity that contributed to a relative decline 
in training. Another factor, according to DOL officials, is that some 
program participants had limited access to needs-based financial 
assistance and other supportive services, such as child care, which 
may have prevented them from entering WIA training services. Further, 
DOL noted that the preference of many program participants is for 
immediate employment rather than job training. 

Similarly, the percentage of training participants who earned 
credentials has also generally declined from program year 2006 through 
program year 2011 (see figure 3). For example, DOL's data show the 
percentage of those who earned credentials in the Adult Program 
dropped from about 74 percent in program year 2006 to about 58 percent 
in program year 2011. Likewise, the percentage of those in the 
Dislocated Worker Program who earned credentials dropped from about 75 
percent in program year 2006 to about 63 percent in program year 2011. 
Despite the decline in the percentage of training participants who 
earned credentials, DOL reported that the total number of participants 
attaining credentials increased during this time period. For example, 
the number of training participants in the Dislocated Worker Program 
who attained credentials increased from about 48,000 in program year 
2006 to about 79,000 in program year 2011. 

Figure 3: Percentage of Training Participants in the Adult and 
Dislocated Worker Programs Who Earned Credentials, Program Years 2006 
through 2011: 

[Refer to PDF for image: multiple line graph] 

Program year: 2006; 
Adult: 74.1%; 
Dislocated Worker: 75.2%. 

Program year: 2007; 
Adult: 70.5%; 
Dislocated Worker: 68.2%. 

Program year: 2008; 
Adult: 65.9%; 
Dislocated Worker: 68.2%. 

Program year: 2009; 
Adult: 60.1%; 
Dislocated Worker: 63%. 

Program year: 2010; 
Adult: 59.2%; 
Dislocated Worker: 62%. 

Program year: 2011; 
Adult: 57.9%; 
Dislocated Worker: 62.9%. 

Source: DOL. 

Note: Data are based on training participants who exited the Adult and 
Dislocated Worker Programs. DOL requires that states report on 
credentials attained by training participants either during WIA 
participation or by the end of the third quarter after the quarter of 
exit from services. 

[End of figure] 

According to DOL officials, the percentage of training participants 
who earned credentials may have declined in part as a result of 
changes in the performance measures that states negotiated. In program 
year 2005, states began requesting waivers to replace the WIA 
performance measures with a set of common performance measures that do 
not include the employment and credential attainment measure for the 
Adult and Dislocated Worker Programs.[Footnote 36] Officials in three 
of the six states we reviewed said that after this request was 
approved, reporting data on credential attainment became a lower 
priority for them. Officials in one of these states also said they 
stopped collecting and reporting these data until DOL issued 
clarifying guidance in 2010 emphasizing the importance of credential 
attainment as a pathway to employment. 

DOL's data on credential attainment also show that participants in the 
Dislocated Worker Program typically have higher credential attainment 
rates than participants in the Adult Program. DOL officials explained 
that the Dislocated Worker Program has a higher funding level that 
supports more training, and that participants in this program 
generally have longer work histories and more advanced education and 
so are more likely to enter into training programs that lead to 
credentials. In contrast, participants in the Adult Program are more 
likely to require training focused on remedial education and job 
readiness which are less likely to result in credentials as defined by 
DOL. 

Of the training participants who attained credentials during program 
year 2011--approximately 89,000 in the Adult Program and 84,000 in the 
Dislocated Worker Program--about two-thirds in each program earned an 
occupational credential, such as a welding certificate or clinical 
medical assistant certificate (see figure 4). The next two most common 
types of credentials attained by training participants were an 
occupational skills license, such as a license in nursing[Footnote 
37], and an associate's degree. The fact that training participants 
attained occupational credentials at higher rates than longer-term 
academic degrees is consistent with DOL's recommendation that states 
shorten training duration in an effort to increase credential 
attainment. In our December 2013 report, we found that in program year 
2011, of those in the Adult Program who entered training, 75 percent 
spent 1 year or less receiving training services, while 25 percent 
spent more time. Similarly, for the Dislocated Worker Program, 65 
percent of training participants spent 1 year or less receiving 
training services.[Footnote 38] According to officials from one local 
workforce investment board we contacted, all training programs offered 
through their training providers must lead to a credential and must be 
completed in 12 months or less. 

Figure 4: Percentage of Credential Types Attained by Training 
Participants in Program Year 2011: 

[Refer to PDF for image: horizontal bar graph] 

High school diploma: 
Adult: 2.1%; 
Dislocated Worker: 0.8%. 

Associate's degree: 
Adult: 8.9%; 
Dislocated Worker: 13.0%. 

Bachelor's degree: 
Adult: 2.2%; 
Dislocated Worker: 1.9%. 

Occupational license: 
Adult: 12.9%; 
Dislocated Worker: 11.7%. 

Occupational credential: 
Adult: 65.2%; 
Dislocated Worker: 65.4%. 

Other type of educational/occupational credential: 
Adult: 8.6%; 
Dislocated Worker: 7.2%. 

Note: Data are based on training participants who exited the Adult and 
Dislocated Worker Programs. 

Source: GAO analysis of DOL data. 

Note: Data are based on training participants who exited the Adult and 
Dislocated Worker Programs. 

[End of figure] 

Little Is Known about the Extent to Which Participants Obtained 
Training-Related Employment: 

We found DOL's data on training-related employment unreliable for our 
purposes based on our analysis of the data, an Office of Inspector 
General report, and data quality reports. We were not able to 
determine how many participants in the Adult or Dislocated Worker 
Programs obtained employment related to their training in program year 
2011. For the Adult Program, we found that states reported data on 48 
percent of training participants, but had missing data for the 
remaining 52 percent. For the Dislocated Worker Program, states 
reported data on 74 percent of training participants, but had missing 
data for the remaining 26 percent. Further, our analysis of the 
reported data showed wide variation among states regarding the 
percentage of participants who obtained training-related employment, 
raising questions about the data's reliability (see table 2). Our 
findings are consistent with a September 2011 Office of Inspector 
General report, which found that DOL's data on training-related 
employment were "incomplete and unreliable."[Footnote 39] 
Specifically, the Inspector General reported that 5 of the 53 state 
workforce agencies it reviewed did not report any data and 12 state 
workforce agencies reported unreasonably high or abnormally low data 
on training-related employment. Further, DOL's WIA data quality report 
for the third quarter of program year 2011 raised questions about 
training-related employment data for 26 states. For example, some 
states reported that none of their training participants secured 
training-related employment. 

Table 2: Variation among State-Reported Percentages of Training 
Participants Who Obtained Training-Related Employment in Program Year 
2011: 

Percentage of training-related employment: 0 to 25%; 
Adult Program[A]: Number of states[C]: 10; 
Dislocated Worker Program[B]: Number of states[C]: 13. 

Percentage of training-related employment: 26% to 50%; 
Adult Program[A]: Number of states[C]: 9; 
Dislocated Worker Program[B]: Number of states[C]: 16. 

Percentage of training-related employment: 51% to 75%; 
Adult Program[A]: Number of states[C]: 10; 
Dislocated Worker Program[B]: Number of states[C]: 23. 

Percentage of training-related employment: 76% to 100%; 
Adult Program[A]: Number of states[C]: 24; 
Dislocated Worker Program[B]: Number of states[C]: 1. 

Source: GAO analysis of DOL data. 

[A] These figures are based on states' reported data on 48 percent of 
training participants and do not include 52 percent of training 
participants, for whom data were missing. 

[B] These figures are based on states' reported data on 74 percent of 
training participants and do not include 26 percent of training 
participants, for whom data were missing. 

[C] Number of states is based on the 53 state workforce agencies that 
reported training-related employment data in program year 2011. 

[End of table] 

Selected States Cited Some Obstacles to Collecting Data on Credential 
Attainment and DOL and States Have Taken Steps to Address Challenges 
and Improve These Data: 

Manual Process of Collecting Data on Credentials Can Be Resource-
Intensive: 

Workforce officials we interviewed in four of six states said that 
collecting data on credential attainment can be resource-intensive 
primarily because it requires manually tracking the information. 
Unlike employment-related outcomes--which states can generally obtain 
through the state's UI wage record system--credentials are not 
generally recorded in a central, automated data system. As a result, 
case managers must manually collect this information from various 
sources including participants, training providers, and third-party 
credentialing organizations. DOL also requires documentation of 
credential attainment with a copy of a diploma, transcript, or other 
approved record in the participant's case file. DOL monitors this 
requirement through its data validation process. The process of 
collecting and verifying a participant's credential attainment 
generally entails one or more of the following steps: 

Contacting Participants. Workforce officials in most states we 
reviewed said they generally begin their efforts to determine 
credential attainment by attempting to contact training participants, 
though some are unresponsive or inaccessible. Several local officials 
noted that they use a variety of means, including phone, mail, email, 
and social media. Some training participants readily provide evidence 
of their credentials. For example, local officials in two states 
estimated that for about 70 percent of participants, credentials are 
fairly easy to verify. Other participants may be less responsive. 
Workforce officials in three states explained that participants who 
have already exited the program have little incentive to respond to 
their requests. Local officials in two states also noted that some 
participants relocate without providing updated contact information. 

Contacting Training Providers. Training providers are another 
potential source of credential information, though in some cases they 
may decline to share such information. If case managers cannot reach a 
participant, they generally contact the training providers to 
determine whether a credential was earned. However, workforce 
officials from three states noted that training providers often 
declined to provide this information, citing student privacy rights 
such as those established by the Family Educational Rights and Privacy 
Act of 1974, as amended (FERPA).[Footnote 40] 

Contacting Third-party Organizations. Third-party credentialing 
organizations represent an additional source of credential 
information.[Footnote 41] For some occupations, a license or 
certification is required before a person can be employed in that 
capacity, such as a licensed practical nurse. In these cases, third-
party organizations, such as state regulatory bodies, issue 
credentials. Case managers can sometimes search licensing databases 
online to confirm credential attainment. Local officials from two 
states noted that such data are fairly easy to obtain because the 
information is generally centrally accessible. However, an official 
from another state said that third-party organizations do not always 
provide information on credential attainment before DOL's reporting 
timeframes end. 

Because case managers may not always be able to track down the 
documentation needed to verify credential attainment, the actual 
number of participants who attain credentials may be underreported to 
DOL. For example, officials in one state we contacted said they 
believe their credential attainment rate should be about 65 percent, 
but the rate they actually report is about half of that. 

Despite such obstacles to verifying data on participant credentials, 
several workforce experts and officials noted the importance of 
collecting this information. Workforce experts from one national 
organization noted that credential attainment can demonstrate the 
value of the funds invested in training and show employers the value 
of workforce programs and their participants. Workforce experts from 
another national organization said that credential data could help 
officials determine which credentials are best aligned with good 
employment outcomes. Some employer groups also noted the value of 
credentials in some high-demand occupations, such as manufacturing and 
information technology. For example, representatives of employer 
groups in Illinois and Rhode Island said they value information 
technology and manufacturing credentials from certifying organizations 
because these programs prepare individuals to perform high-skill tasks. 

To Address Challenges, DOL Has Stressed Both Credential Attainment and 
Reporting: 

In 2010, DOL provided guidance to states to increase the quantity and 
quality of credentials attained and to clarify the definition of 
credential for reporting purposes. During early WIA implementation, we 
reported that the definition of credential varied within and across 
states. For example, some states strictly defined credential as a 
diploma from an accredited institution, and other states broadly 
defined credential to include certificates of job readiness or 
completion of a workshop.[Footnote 42] DOL issued guidance in 2006 
that provided additional clarification on which credentials to report, 
but, according to some workforce officials and experts, allowed for 
some interpretation.[Footnote 43] In 2010, DOL issued guidance that 
defined "credential" as an umbrella term that can include a range of 
postsecondary degrees, diplomas, licenses, certificates, and 
certifications.[Footnote 44] DOL also clarified that credentials must 
show attainment of measurable technical or occupational skills 
necessary to obtain employment or advance within an occupation. For 
this reason, DOL specified that credentials related to remedial 
training, such as work-readiness certificates, would not be counted 
for the purposes of credential attainment. 

In addition to clarifying which credentials should be reported, the 
2010 guidance also included strategies that state and local officials 
can use to increase the quantity and quality of credentials attained. 
It noted that the first step in increasing the quantity of credentials 
attained is to refer more participants to training. DOL's guidance 
also encouraged officials to take steps to ensure that the training 
programs result in an industry-recognized credential and that 
participants complete these training programs. These steps include 
shortening the duration of training and providing supportive services 
that enable participants to succeed. Further, to improve the quality 
of credentials attained, DOL suggested that state and local agencies 
build the capacity of front-line staff to identify and assess valuable 
and appropriate credentials for participants. 

DOL has also stressed the importance of credential attainment by 
measuring it through an agency-wide performance goal for its workforce 
development programs including the WIA Adult and Dislocated Worker 
Programs. DOL officials noted that credential attainment rates for 
these WIA programs are higher than the rates of some other DOL 
programs included in the agency-wide performance goal. DOL first began 
tracking credential attainment data for its agency-wide performance 
goal in 2010 when it set out to increase the number of training 
participants who attain credentials through any one of multiple 
federal workforce programs. Specifically, the goal was an increase of 
10 percent, up to a total of 220,000 training participants earning 
credentials. In fiscal year 2013, DOL continued to assess credential 
attainment through this performance goal and sought to increase the 
percentage of training participants who earn credentials from 57 to 62 
percent.[Footnote 45] DOL officials reported a credential attainment 
rate of 59.4 percent through the first two quarters of fiscal year 
2013. DOL officials also said that DOL has established a new 
credential attainment goal; specifically, that by September 30, 2015, 
the percentage of training participants who attain credentials will 
increase by 10 percent from the level reported as of the end of fiscal 
year 2013. 

In addition to issuing guidance and setting credential attainment 
goals, DOL also undertook a number of other related initiatives, 
including some that are specific to credentials and others that are 
more broadly designed, such as the Workforce Data Quality Initiative. 
See Table 3 for a description of DOL's initiatives. 

Table 3: Key DOL Initiatives Related to Credential Attainment: 

Guidance to workforce system: 

Initiative: Issued guidance that more precisely defined credentials for 
purposes of WIA reporting; and in 2010 provided strategies state and 
local workforce agencies, partners, and grantees could take to increase 
the rate of credential attainment among workforce program participants 
and improve the quality of those credentials; 
Date: February 2006 and December 2010. 

Agency-wide initiatives: 

Initiative: Set credential attainment as an agency-wide performance 
goal across various workforce programs, including the WIA Adult and 
Dislocated Worker programs; 
Date: 2010-2015. 

Initiative: Created a credentials database to disseminate information 
on available credentials and Certification Finder website to help 
employers identify occupational certifications that may be useful for 
current or future employees; 
Date: 2005-ongoing. 

Initiative: Held Credentials Forum for employers to identify 
credentials valued by employers and updated the credentials database; 
Date: 
May 2012-January 2013. 

Initiative: Began Career Pathways Initiative (jointly with Education 
and Health and Human Services) to help adults acquire marketable skills 
and industry-recognized credentials through better alignment of 
education, training and employment, and human and social services among 
public agencies and with employers; 
Date: 2010 -ongoing. 

Longer-term initiatives: 

Initiative: Awarded Workforce Data Quality Initiative grants to states 
to help them link their statewide educational databases to their 
statewide workforce databases and improve access to data in their 
workforce databases, including data on credentials; 
Date: 2011-2013. 

Source: GAO summary of DOL information. 

[End of table] 

Selected States and Local Areas Have Taken Steps to Address Challenges 
by Enhancing Attainment and Reporting of Credentials: 

Some states have stressed the importance of credential attainment by 
implementing broad, statewide efforts. Similar to DOL's efforts to 
enhance credential attainment by establishing annual goals, three of 
the six states we reviewed have either implemented statewide 
credential attainment goals or are working to do so: 

* Texas implemented an annual state performance measure on educational 
achievement that tracks credential attainment for multiple programs, 
including the WIA Adult and Dislocated Worker Programs. All WIBs in 
the state are held to this measure. 

* Washington has made credential attainment a state performance 
measure, but defines credential more broadly than DOL. For example, 
Washington recognizes a larger range of credentials, such as 
completion of on-the-job training.[Footnote 46] 

* Illinois re-implemented a credential attainment performance measure 
during program year 2012 and, according to state officials, is in the 
process of setting credential targets for program year 2014. 

According to officials from some states, their efforts to emphasize 
credential attainment and reporting may have a positive impact on 
participants' reported rate of credential attainment. 

Moreover, officials in Alabama, Illinois, Kansas, and Rhode Island 
told us they targeted their training funds more narrowly on credential-
yielding programs by only approving training providers with programs 
that resulted in credentials that met DOL's definition. For one 
Chicago WIB, this strategy, along with its other efforts to streamline 
training options from 753 occupations to 40 in-demand occupations, 
reduced its number of training providers. This practice was one of 
many DOL suggested in its 2010 guidance as a means for states and 
local areas to improve the value of credentials for participants. 
Officials in nearly every state we interviewed reported that this 
guidance was helpful largely because it more clearly defined which 
credentials should be reported to DOL. 

In addition, selected states and local areas have taken steps to ease 
the resource-intensive process of collecting data on credentials by 
enhancing communication with participants and working to overcome 
privacy issues with training providers. Workforce officials in three 
states told us that case managers seek to build rapport with 
participants early in the process so they are more likely to be 
responsive after their training program ends. Regarding training 
providers, officials in four of six states said they have made efforts 
to address privacy concerns. Local officials in Alabama, Kansas, and 
Texas, for example, told us that they ask participants to sign consent 
forms to allow training providers to share credential information with 
officials. In Washington, state officials access some credential data 
from the National Student Clearinghouse and from their state database 
of community and technical colleges.[Footnote 47] They said that 
student privacy rights are generally not a barrier to accessing 
credential data in Washington because students attending their 
community and technical colleges are notified that such information 
can be released to other entities unless the student opts out of 
sharing it. Washington state officials noted that they have been 
refining their process for collecting data on credential attainment 
for 15 years and now have a fairly sophisticated approach. 

While these varied efforts to mitigate challenges may help reduce the 
resources required or improve the quality of reported data, workforce 
officials from three states and three experts we interviewed raised 
some additional considerations about measuring performance on 
credential attainment (see table 4). Establishing a performance 
measure on credential attainment may affect the type of training 
provided and which participants receive training. For example, neither 
work readiness training nor on-the-job training (OJT) leads to what 
DOL has defined as a credential for reporting purposes. However, these 
may be the most appropriate types of training for participants with 
basic skills or for particular industries, according to officials from 
two states we interviewed.[Footnote 48] Our December 2013 report found 
that participants in the WIA Adult and Dislocated Worker Programs 
often lacked the relevant qualifications and basic skills needed to 
participate in training that would meet the needs of employers seeking 
employees for in-demand occupations.[Footnote 49] In addition, work 
readiness certificates are generally valued by employers, according to 
several employer group representatives and local workforce officials 
we interviewed.[Footnote 50] Representatives from a few employer 
groups also noted that, in some cases, experience is more important 
than credentials. For example, local officials we interviewed in 
Illinois said that the vast majority of participants in OJT obtained 
jobs with the employers once their training was completed. The 
officials said OJT provided a good return on investment, despite the 
fact that these participants did not earn credentials.[Footnote 51] 
Currently, DOL's credential attainment data do not include 
participants who completed these types of training programs. If 
credential attainment is established as a performance measure, it will 
be important to consider ways to address participants who are enrolled 
in certain types of training that do not lead to a credential, such as 
by excluding these participants from a credential attainment measure 
or considering if other measures, such as basic skill attainment, 
could capture the value of training provided to participants excluded 
from the measure. 

Table 4: Establishing a Performance Measure on Credential Attainment: 
Challenges, Mitigation Strategies, and Considerations: 

Challenges: Collecting data can be resource-intensive; 
Potential mitigation strategies: 
* Build relationship with participant early on to enhance 
responsiveness to follow-up efforts; 
* Build relationship with training provider to enhance responsiveness 
to follow-up efforts; 
* Have participants waive student privacy rights so training providers 
can share credential information; 
* Seek automated or direct access to centralized credential databases; 
Related considerations: 
* Strategies do not guarantee a reduction in the resources required 
because case managers must still follow up with participants and 
training providers; 
* Because data collection can be dependent on multiple data sources 
and the cooperation of participants and training providers, these 
strategies do not guarantee that local areas will be able to document 
credential attainment for all participants. This may result in 
underreporting. 

Challenges: Some training does not result in a DOL defined credential, 
but may still be needed by participants and valued by employers; 
Potential mitigation strategies: 
* Exclude participants who are not in training that potentially leads 
to a DOL-defined credential from the calculation of the measure, by 
excluding those in OJT, for example; 
Related considerations: 
* If such participants are not excluded, it could 
create a disincentive to offer some types of training or to serve 
certain population groups with limited skills; 
* Consider if other measures, such as basic skill attainment, could 
capture the value of training provided to participants excluded from 
the measure. 

Source: GAO analysis of interviews with selected state and local 
officials and workforce experts. 

[End of table] 

States Face Continuing Challenges Reporting Data on Training-Related 
Employment, Despite Efforts by States and DOL to Improve Reporting: 

Reporting Data on Training-Related Employment Is Resource-Intensive 
and Involves Subjective Decisions: 

Workforce officials in most states we studied identified challenges 
reporting data on training-related employment that were greater than 
those for reporting data on credential attainment, including the high 
degree of resources required and the subjective nature of determining 
whether employment is linked to training. Similar to credential 
attainment, there is no definitive source for these data, so case 
managers must generally collect participants' employment information 
from various sources, including participants, employers, and UI wage 
records. Then--in a step beyond what is required for reporting on 
credential attainment--they must piece this information together to 
determine whether participants' employment is substantially related to 
their training. Also unlike reporting on credential attainment, DOL 
does not require that local WIA programs collect and retain 
documentation on training-related employment in the participants' case 
files to verify the accuracy of data they report to states.[Footnote 
52] 

Officials in four of the six states we studied, as well as at DOL, 
said this data collection process often requires considerable time and 
effort. Further, officials from DOL and four states emphasized the 
need to consider the balance between the time required to collect 
outcome data and the time case managers spend serving participants, 
especially in an environment of reduced resources. In addition, 
officials in all six states said that making training-related 
employment determinations can be subjective. According to DOL's 
reporting guidance, participants' employment is related to their 
training if it uses "a substantial portion of the skills taught in 
training."[Footnote 53] However, officials in all six states, as well 
as workforce experts from one national organization, said that one 
case manager's interpretation of what constitutes a substantial 
portion of the skills obtained in training may differ from another's. 
The training-related employment decision can be straightforward if the 
training and job are clearly connected. For example, if a participant 
received training to attain a commercial driver's license and was 
subsequently hired as a driver by a trucking company, the case manager 
can easily determine that the participant's employment is 
substantially related to the training received. In other cases, 
however, the decision may be more subjective. For example, officials 
in one state could not agree whether a participant who had received 
aviation instruction training had secured training-related employment 
in his position as an airframe and power plant mechanic. Some state 
officials thought the skills obtained were transferable, but others 
were unsure. 

Collecting participants' employment information and attempting to 
determine whether it was training-related generally entails several 
steps (see figure 5). 

Figure 5: General Process for Collecting Participants' Employment 
Information and Determining Whether It Relates to their Training: 

[Refer to PDF for image: illustration] 

Case manager seeks employment information: 

Case manager may make repeated attempts: 
Former participant; 
Employer; 
Unemployment Insurance (UI) wage records. 

If employment information is obtained, case manager attempts to 
determine whether employment is training related. 

Source: GAO interviews with local workforce officials. 

[End of figure] 

Contacting participants. Several state and local officials we 
interviewed said that they generally begin the process of collecting 
data on training-related employment by attempting to contact 
participants, though some can be unresponsive or inaccessible, which 
workforce experts from one national organization noted as well. If 
case managers are successful, they ask participants for information 
such as the name of their employer and their job title. Some local 
areas also ask participants directly if their new jobs are--in the 
participants' opinion--related to the training they received. In some 
cases, case managers make their training-related employment 
determinations based solely on information the participants provide 
about their employment. 

Contacting employers. Some case managers contact employers to obtain 
participants' employment information, though employers may not be 
responsive. If case managers could not reach a participant but know 
where the person works, they may contact the employer to obtain their 
job title and description. They may also contact an employer to verify 
the information provided by a participant. Case managers can use the 
employment information obtained from an employer, or from both the 
participant and employer, to determine if a participant's job is 
training-related. However, workforce officials in Illinois and Texas 
said employers may not be responsive because they are concerned about 
employee privacy or about the amount of follow-up required. 

Checking UI wage records. Some local workforce officials said that if 
they are unable to gather information about a participant's employment 
from the participant or the employer, they check the UI wage records, 
which are generally not available until several months after a 
participant exits from WIA services.[Footnote 54] DOL officials and 
workforce experts at two national organizations said the UI wage 
records generally provide the name of the participant's employer and a 
code associated with the employer's industry, but specific information 
on the participant's occupation is rarely included. In some cases, the 
industry code has a clear connection to the training received, making 
the case manager's training-related employment determination 
straightforward. For example, if a participant who was trained as a 
nurse was hired by a hospital, the case manager can reasonably assume 
that the employment and training are related. However, some officials 
noted that the industry code is not always a good predictor of a 
participant's occupation. For example, if the same participant was 
hired by the health unit of a manufacturing company, the industry code 
in the UI wage records would suggest that the person's job was 
associated with manufacturing and not related to the nursing training. 

Other steps to determine training-related employment. If successful in 
accessing a participant's employment information, including job title, 
some workforce officials said case managers use DOL's Occupational 
Information Network (O*NET), which provides an online tool to match 
job title occupational codes to the skills code associated with the 
participant's training. This can help case managers decide whether the 
participant's employment is training-related.[Footnote 55] 

According to some local workforce officials, however, it can be 
difficult to find the precise occupational code that matches the 
participant's new job. In addition, DOL officials told us that even 
attempting to match O*NET codes in this manner might not help case 
managers determine if participants' employment is related to their 
training because the threshold of relatedness is still subjective, as 
mentioned previously. DOL officials also told us it is difficult to 
prescribe a standard definition for determining whether a job is 
related to training because it often requires some judgment on the 
part of local officials. 

States Have Taken Steps to Address Challenges by Improving Reporting 
on Training-Related Employment: 

Side bar: 

Washington state collects additional information on training-related 
employment: 

In addition to reporting data on training-related employment to DOL, 
Washington state surveys a subset of WIA participants three quarters 
after exit and asks them about: 

* the extent to which their new job is related to their training; 

* the extent to which the training they received helped them get their 
job, and; 

* how useful the skills they learned in training are in doing their 
job. 

Officials said they conduct this survey for internal use to collect and 
share data on training-related employment that is not otherwise 
available to program managers and state officials. Another official 
noted that a significant percentage of respondents who do not report 
their job as training-related find the training instrumental in getting 
the job or that the skills they acquired in training are useful on 
their job. 

Source: Interview with state officials in Washington and survey 
documents. 

[End of side bar] 

To improve reporting, some states have taken steps to increase their 
access to information about participants' employment. Similar to their 
efforts to collect information on credential attainment, some local 
workforce officials told us that case managers seek to build rapport 
with participants. At the same time, officials we interviewed in four 
of six states have taken steps to increase access to employment-
related information. In Illinois, Rhode Island, and Texas, local 
workforce officials ask participants to sign release forms authorizing 
employers to release employment information to officials. Local 
workforce staff in Kansas and Texas also said they obtain information 
on participants' employment, including the names of their employers, 
and--unlike UI wage records--their job titles, by subscribing to an 
online payroll database called The Work Number. This service verifies 
employment via a database of national payroll data but does not 
include all employers, and local WIA programs must pay to subscribe. 

In addition, local workforce officials in four states we contacted 
said they have developed strategies to help reduce the subjectivity in 
determining whether a participant's employment is related to their 
training. For example, local officials in Texas told us that staff may 
consult their American Job Center's local business services office, 
which often has specific knowledge about what skills correspond with 
particular job titles. Officials in Texas and Alabama also said case 
managers may consult with their peers or supervisors to reach 
consensus about a training-related employment decision. Further, state 
workforce officials in Kansas told us that when employers post jobs on 
the state's job bank, they are required to enter occupational codes 
from DOL's online O*NET database. If a training participant gets one 
of these jobs, case managers can compare the employer-provided 
occupational code with the training codes supplied by training 
providers to help them determine whether the job is training-related. 
State and local officials in Washington also said managers routinely 
use O*NET when making their training-related employment decisions. 

DOL Has Taken Limited Steps to Address Challenges and Improve Data on 
Training-Related Employment: 

While DOL has recently issued guidance aimed at increasing reporting 
rates for training-related employment data, it has taken limited steps 
to address states' ongoing reporting challenges. As previously 
discussed, a September 2011 report by DOL's Office of Inspector 
General raised questions about the quality of these data and made 
recommendations to address this issue.[Footnote 56] In response, DOL 
issued a notice to states in September 2013 that reiterated the 
requirement for states to report these data and stressed the data's 
importance for program analysis and evaluation efforts pertaining to 
the value of investments in WIA training.[Footnote 57] The DOL notice 
also acknowledged that reporting on training-related employment is 
challenging largely because the information must be collected 
manually. According to the DOL notice, nearly every state that 
participated in conference calls on the topic indicated that there was 
considerable cost in conducting the necessary follow-up for reporting 
on training-related employment and that this was the primary reason 
the data were not well-reported. State and local officials we 
interviewed also noted that such data may be underreported because of 
the difficulty of following up with participants and employers. DOL 
also concluded that states with larger training caseloads had less 
complete reporting on training-related employment. In the notice, DOL 
also described a few practices most common among states with higher 
reporting rates for training-related employment.[Footnote 58] For 
example, DOL cited the practice of instituting a data system check to 
ensure that training-related employment data are recorded before 
closing a participant's case file. 

Side bar: 

Use of crosswalk not necessarily a solution for determining training-
related employment: 

Another program DOL administers, Job Corps--a residential, 
educational, and career technical training program for disadvantaged 
youths--uses a crosswalk that links specific training codes and 
occupational codes to help staff determine training-related 
employment. However, in a September 2011 report, the DOL Inspector 
General found that this crosswalk included some matches that were 
either not related or poorly related. Moreover, DOL officials said that 
using a crosswalk for the WIA Adult and Dislocated Worker programs 
could make the training-employment link too restrictive and would 
require a considerable amount of resources to develop. 

Source: Interviews with DOL officials and U.S. Department of Labor 
Office of Inspector General Audit Report No. 26-11-004-03-370, Job 
Corps Needs to Improve Reliability of Performance Metrics and Results, 
September 30, 2011. 

[End of side bar] 

DOL has not identified and disseminated strategies for increasing 
access to employment-related data or helping to minimize the 
subjectivity of training-related determinations, but instead has 
focused exclusively on increasing data reporting rates. While DOL 
officials maintain that manual follow-up with participants is the best 
approach for obtaining employment related data, they also recognize 
that such data collection is resource-intensive. In addition, as 
officials in all six of our selected states also noted, DOL officials 
acknowledged that determining whether a participant's employment is 
training-related can be a subjective decision. They noted challenges 
in defining training-related employment more precisely. For example, 
officials said some training is intended to develop broad, nonspecific 
skills that may help participants get jobs but are not associated with 
a specific occupation or industry. We recognize that utilizing 
professional judgment is inherent in certain tasks such as determining 
whether a participant's employment is related to the training the 
participant received. However, minimizing the amount of subjective 
decision-making involved to the extent possible could help ensure 
better quality data on training-related employment. Reasonable 
approaches for improving the quality of performance data focus on 
aspects of completeness, accuracy, consistency, and ease of use. 
[Footnote 59] By identifying and sharing with states practices to 
increase access to employment-related data and reduce the subjectivity 
of some determinations, DOL could help states improve their reporting 
of data on training-related employment. 

In addition to the strategies all six selected states use to mitigate 
reporting challenges, workforce officials from three states and 
workforce experts at two national organizations said some additional 
considerations should be taken into account in weighing a performance 
measure on training-related employment (see table 5). We previously 
noted concerns about the level of resources required and the 
subjectivity of determinations, both of which could affect the data's 
completeness and consistency--key aspects of performance data quality. 
The state and local strategies we identified may help mitigate some 
reporting challenges. In addition, a participant's successful 
placement in a training-related job depends on both the ability and 
decision to pursue such a position. Some local workforce officials we 
contacted said that it may take participants longer to find employment 
in their field of training than is allowed for reporting purposes. 
[Footnote 60] For example, workforce officials in Rhode Island said 
that due to the poor economy in their state, it is not uncommon for 
some participants to take 2 years or more to find a job. Additionally, 
workforce officials in three states we contacted said participants may 
decide to take a job unrelated to their training if it is the only job 
they can find or if they simply choose not to pursue a job in the 
field in which they were trained. 

Table 5: Establishing a Performance Measure on Training-Related 
Employment: Challenges, Potential Mitigation Strategies, and 
Considerations: 

Challenge: Collecting data can be resource-intensive; 
Potential mitigation strategies: 
* Build relationship with participant early on to enhance 
responsiveness to follow-up efforts; 
* Have participants sign release forms so employers can share 
credential information; 
* Subscribe to payroll record databases (e.g. The Work Number); 
* Require employers to report occupational codes to state job banks; 
Related considerations: 
* Strategies do not guarantee a reduction in the resources required to 
collect data because case managers must still follow-up with 
participants and employers; 
* Because data collection can depend on multiple data sources and the 
cooperation of participants and employers, these strategies do not 
guarantee that local areas will be able to obtain necessary employment 
information. Participants and employers may be unresponsive, or 
employers may not report complete data on their employees. This may 
result in underreporting; 
* State and local workforce agencies may not have the resources to 
subscribe to payroll record databases; 
* Requiring employers to report occupational codes could place more 
burdens on employers. 

Challenge: Determining whether employment is related to training is 
subjective; 
Potential mitigation strategies: 
* Consult the American Job Center's local business services office; 
* Discuss internally and reach consensus with supervisors or peers; 
* Use O*NET to find occupational codes associated with job titles and 
match to training code; 
Note: The strategies presented in the row above may also help mitigate 
the subjectivity of determining training-related employment; 
Related considerations: 
* Additional input may not help make a valid determination; 
* Using O*NET to find specific occupational codes and matching to 
training codes may still involve some subjectivity; 
* It is generally straightforward to make training-related 
determinations for some types of training such as OJT and customized 
training for employers. 

Challenge: Participants may not find a training-related job before the 
reporting deadline; 
Potential mitigation strategies: 
* Provide job search and placement assistance to help participants 
find training-related jobs; 
Related considerations: 
* Strategies do not guarantee a successful placement in a training-
related job. 

Challenge: Participants may choose to take a job unrelated to their 
training; 
Potential mitigation strategies: 
* Counsel participants about related jobs before placing them in 
training to help ensure the job they are training for is of interest 
to them; 
Related considerations: 
* Participants may still decide to take a job unrelated to their 
training. 

Source: GAO analysis of interviews with selected state and local 
officials and workforce experts. 

[End of table] 

Conclusions: 

To ensure that public funds invested in WIA's Adult and Dislocated 
Worker Programs are spent wisely, program managers and policymakers 
need performance data that are accessible, complete, accurate and 
consistent. The current common performance measures--employment, 
retention, and earnings--provide a basis for assessing the overall 
value of the services the programs provide, primarily using a 
standardized data source (state UI wage records). Beyond these, data 
on outcomes such as credential attainment and training-related 
employment can potentially provide information more specifically on 
the value of training services. However, as we have noted, collecting 
data on these outcomes can be more resource-intensive, in part because 
there is no single readily available source of data. 

DOL has taken steps to elevate the importance of credential attainment 
and improve data quality for this outcome. We found credential 
attainment data reported to DOL to be reliable. In contrast, we found 
the data reported to DOL on training-related employment to be 
incomplete and inconsistent. While DOL has acknowledged challenges in 
collecting data on and determining training-related employment, it has 
taken only limited steps to address these challenges, focusing efforts 
exclusively on improving reporting rates. This effort alone will not 
improve the quality of the data being reported. Given the nature of 
the challenges we identified, there are no easy or complete solutions. 
However, we also identified strategies some states use that may help 
increase access to employment information and reduce the subjectivity 
of some training-related determinations. Sharing such strategies with 
other states, as well as identifying and communicating other 
approaches, could lead to incremental improvement in the quality of 
data reported. Without such action, the data states are required to 
report on training-related employment are likely to remain unusable. 

Recommendation for Executive Action: 

To provide policymakers and program managers with better quality 
information to assess the value of training provided by WIA's Adult 
and Dislocated Worker Programs, we recommend that the Secretary of 
Labor identify and share with states strategies for collecting and 
reporting data on training-related employment that could either 
increase access to employment information or reduce the subjectivity 
of determining when training is related to employment. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to the Secretary of Labor and 
selected draft sections to the Secretary of Education. DOL and 
Education provided technical comments, which we incorporated as 
appropriate, and DOL provided a written response (see appendix II). 
DOL agreed with our recommendation and noted that having reliable data 
on training-related employment is important to effectively manage and 
evaluate the Adult and Dislocated Worker Programs. DOL also agreed 
that states can benefit from learning what other states are doing to 
address challenges regarding access to and subjectivity of these data. 
Toward this end, DOL noted that it plans to conduct additional 
conference calls with state officials to reiterate the importance of 
identifying training-related employment and continue to discuss and 
share best practices to improve these data. DOL noted that this 
sharing of best practices would supplement actions it has already 
taken to improve data on training-related employment. These actions 
include coding changes to the WIASRD to capture additional 
information, conference calls with state workforce officials to 
discuss reporting on training-related employment, and a work group 
considering adding more data elements to the UI wage records such as 
an occupational code. 

As agreed with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
from its issue date. At that time, we will send copies of this report 
to the appropriate congressional committees, the Secretary of Labor, 
the Secretary of Education, and other interested parties. In addition, 
the report is available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-7215 or sherrilla@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who make key contributions 
to this report are listed in appendix III. 

Signed by: 

Andrew Sherrill: 
Director: 
Education, Workforce,
and Income Security Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives for this study on the Workforce Investment Act's (WIA) 
Adult and Dislocated Worker Programs were to examine: 1) the extent to 
which training participants obtain credentials and secure training-
related employment, 2) challenges states face reporting on credential 
attainment and what steps, if any, they and DOL are taking to address 
them, 3) challenges states face in reporting on training-related 
employment and what steps, if any, they and DOL are taking to address 
them. 

To address these objectives, we reviewed relevant federal laws, 
regulations, and DOL's guidance to states for reporting select data on 
participants in the WIA Adult and Dislocated Worker Programs. We 
interviewed DOL officials from the Employment and Training 
Administration and the Office of Inspector General as well as experts 
on workforce issues (see Selection of Experts, below). We also 
interviewed state and local workforce officials as well as 
organizations that provided an employer perspective from a 
nongeneralizable sample of six states (see Selection of States and 
Local Areas, below). To assess the reliability of the data DOL 
collects on credential attainment and training-related employment, we 
analyzed data from the Workforce Investment Act Standardized Record 
Data (WIASRD) system for program year 2010 and program year 2011--the 
most recent years for which data were available--by testing the data 
electronically and interviewing knowledgeable agency officials. 
[Footnote 61] We found the data to be sufficiently reliable for 
providing estimates on: 1) participants who received training, 2) the 
minimum number of training participants who earned a credential, and 
3) the types of credentials they earned. However, we determined that 
the data on training-related employment were not reliable for the 
purposes of our report (see Analysis of DOL's Training Outcome Data 
below). 

Selection of Experts: 

We interviewed experts on workforce issues representing six 
organizations. We identified experts by first reviewing relevant 
literature and asking officials from DOL for recommendations. We 
further developed the list by asking each expert we interviewed for 
additional names. Only experts who were mentioned more than once were 
selected. The results of these interviews are not generalizable, but 
provided insights about the overall challenges states face in 
reporting on these outcomes and any efforts by states to overcome them. 

Selection of States and Local Areas: 

We interviewed state and local workforce officials from a 
nongeneralizable sample of six states. We conducted in-person 
interviews with officials in Alabama, Illinois, and Texas and 
telephone interviews with officials in Kansas, Rhode Island, and 
Washington (see table 4). These results are not generalizable, but 
provided insights about the challenges they face in reporting data on 
credential attainment and training-related employment as well as any 
steps they are taking to address those challenges. We selected the 
states to reflect a mix of those that had leading-edge data practices 
(as identified by experts) and those with either incomplete data or 
relatively high rates of reporting errors (as identified by WIA data 
quality reports on credential attainment).[Footnote 62] We also 
factored geographic diversity into state selection. In each state, we 
interviewed state workforce officials and also local workforce 
officials from at least one workforce investment board and at least 
one American Job Center--formerly known as a one-stop center. We 
selected a non-generalizable sample of local areas based on input from 
state workforce officials and, for states we visited, proximity to the 
state workforce agency. In addition, we interviewed at least one 
employer organization in each state. In selecting these entities, we 
considered states' input on organizations that could provide us with 
an employer perspective on the value of credentials and certain types 
of training for various industries. They included statewide business 
associations, regional business associations, individual employers, 
and industry-specific representatives. 

Table 6: Selected Site Visit States and Local Areas: 

State: Alabama; 
Workforce Investment Board: Alabama Workforce Investment Area; 
City of American Job Center: Montgomery. 

State: Illinois; 
Workforce Investment Board: Chicago Cook Workforce Partnership; 
City of American Job Center: Chicago. 

State: Kansas; 
Workforce Investment Board: Workforce Alliance of South Central Kansas; 
City of American Job Center: Wichita. 

State: Rhode Island; 
Workforce Investment Board: Workforce Solutions of Providence/Cranston; 
City of American Job Center: Providence. 

State: Texas; 
Workforce Investment Board: Workforce Solutions: Rural Capital Area; 
City of American Job Center: Round Rock. 

State: Texas; 
Workforce Investment Board: Workforce Solutions: Alamo; 
City of American Job Center: San Antonio. 

State: Washington; 
Workforce Investment Board: Workforce Development Council of Seattle-
King County; 
City of American Job Center: Seattle. 

Source: GAO state and local areas interviews. 

[End of table] 

In each state, we obtained information about the state and local 
area's process of collecting data on credentials and employment 
related to training as well as any challenges they may have 
encountered. We also asked state officials about DOL's related 
guidance. We used a semi-structured interview guide for the state and 
local interviews. 

Analysis of DOL's Training Outcome Data: 

To assess the reliability of DOL's data on training, credential 
attainment, and training-related employment in the WIASRD database for 
participants in the WIA Adult and Dislocated Worker programs, we: (1) 
reviewed documentation related to reporting these data, including 
DOL's Office of Inspector General reports; (2) tested the data 
electronically to identify potential problems with consistency, 
completeness, or accuracy; and (3) interviewed knowledgeable DOL 
officials about the data. Our electronic testing consisted of 
identifying inconsistencies, outliers, and missing values. In 
addition, we analyzed the publicly-available WIASRD data file for 
program years 2010 and 2011, which was produced for DOL by its data 
contractor, Social Policy Research Associates. As part of our 
analysis, we reviewed the steps the data contractor took to address 
data errors and, to the extent possible, compared the data DOL 
provided for our analysis to the publicly-available file, and found 
only slight discrepancies. We found the data on training and 
credential attainment to be sufficiently reliable for reporting 
estimates of: (1) participants who received training, (2) the minimum 
number of training participants who earned a credential, and (3) the 
types of credentials they attained. We were not able to reliably make 
state-to-state comparisons because two states are piloting a new 
reporting format for DOL, and therefore would not have been compatible 
with the others. For the purposes of this report, we did not find the 
data on training-related employment reliable. We reached this 
conclusion based on our analysis of the data, an Office of Inspector 
General report, and DOL's data quality reports. We were not able to 
determine how many training participants in the Adult or Dislocated 
Worker Programs obtained employment related to their training in 
program year 2011. For the Adult Program, we found that states 
reported data on 48 percent of training participants, but had missing 
data for the remaining 52 percent. For the Dislocated Worker Program, 
states reported data on 74 percent of training participants, but had 
missing data for the remaining 26 percent. Further, an analysis of the 
reported data showed wide variation among states regarding the 
percentage of participants who obtained training-related employment 
raising questions about the data's reliability. 

We conducted this performance audit from October 2012 to January 2014 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit work to 
obtain sufficient, appropriate evidence to provide a reasonable basis 
for our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Labor: 

U.S. Department of Labor: 
Employment and Training Administration: 
200 Constitution Avenue, N.W. 
Washington, D.C. 20210: 

January 24, 2014: 

Andrew Sherrill: 
Director: 
Education, Workforce, and Income Security Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, District of Columbia 20548: 

Dear Mr. Sherrill: 

On behalf of the U.S. Department of Labor (the Department), I want to 
thank you for the opportunity to review the Government Accountability 
Office's (GAO) draft report entitled: Workforce Investment Act: 
Strategies Needed to Improve Certain Training Outcome Data (GAO-14-
137). We appreciate the information provided in the report and the 
opportunity to comment. 

The GAO report makes one recommendation to the Department: 

To provide policy makers and program managers with better quality 
information to assess the value of training provided by WIA 's Adult 
and Dislocated Worker programs, we recommend that the Secretary of 
Labor identify and share with states strategies for collecting and 
reporting data on training-related employment that could either ease
access to employment information or reduce the subjectivity of 
determining when training is related to employment. 

In response to this recommendation, the Department's Employment and 
Training Administration (ETA) will conduct additional conference calls 
with staff of state workforce agencies to reiterate the importance of 
identifying training-related employment and to discuss and share best
practices, and updates on how states have been accomplishing this 
task. The Department understands the importance of reliable data on 
training-related employment and appreciates the attention that this 
report brings to states' challenges in collecting and reporting these 
data. The Department agrees that states can benefit from learning what 
other states are doing to address these challenges regarding access to 
and subjectivity of these data. 

This sharing of best practices would supplement a number of actions 
that the Department has already taken to improve data on training-
related employment, which includes the following: 

* Held conference calls with states in August 2012 regarding the topic 
of the reporting of training-related employment in the Workforce 
Investment Act Standardized Record Data (WIASRD). 

* Issued ETA Training and Employment Notice 5-13, Reporting Training-
Related Employment under the Workforce Investment Act (WIA) Programs 
(September 2013), which is described in the GAO report and includes 
best practices identified in the August 2012 conference calls. 

* Added a "Not Known" option for the WIASRD training-related 
employment field, effective July 2013, to remove ambiguity between 
negative versus unknown responses. 

* Added to WIASRD an industry code of employment field, effective July 
2013, using the North American Industry Classification System code 
that is usually available through state wage records. 

* Participates in a Wage Record Enhancement Work Group, convened by 
the Workforce Information Council in cooperation with the National 
Association of State Workforce Agencies, since March 2013. This work 
group is tasked with studying the potential benefits to end-users and 
costs to employers of adding more data elements (such as an 
occupational code) to the state Unemployment Insurance quarterly wage 
records. 

We agree that ensuring the reliability of training-related data 
reported by the states is important to effectively manage and evaluate 
Workforce Investment Act (WIA) Adult and Dislocated Worker programs. 
However, we believe that such improvement should be balanced with the
flexibility that WIA gives to states and with the Department's 
responsibility to prioritize use of its limited resources. Similar to 
GAO, the Department recognizes that training-related employment 
inherently has a certain level of sUbjectivity associated with the 
metric. We have worked with states to ensure that quality data are 
collected for training-related employment under WIA. Calls with state 
workforce professionals in August 2012 confirmed that this data 
element is often under-reported due simply to the high resource burden 
associated with its collection - a burden noted in the GAO report. 
Because there currently is no cost-effective method available to 
obtain the occupation of employment without using manual follow-up, the
Department believes that the most reliable information is already 
being collected. Nonetheless, as noted earlier, the Department will 
continue to communicate with state workforce agencies about the 
importance of and strategies for improving data on training-related 
employment. 

If you would like additional information, please do not hesitate to 
call me at (202) 693-2700. 

Sincerely, 

Signed by: 

Eric M. Seleznow: 
Acting Assistant Secretary: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov. 

Staff Acknowledgments: 

In addition to the contact named above, Laura Heald, Assistant 
Director, John Lack, Jeffrey G. Miller, and Kathryn O'Dea Lamas made 
key contributions to this report. Also contributing to this report 
were James Bennett, Jessica Botsford, David Chrisinger, Elizabeth 
Curda, Kathy Leslie, Carol Patey, Rhiannon Patterson, Jerry Sandau, 
and Walter Vance. 

[End of section] 

Related GAO Products: 

Workforce Investment Act: Local Areas Face Challenges Helping Employers 
Fill Some Types of Skilled Jobs. [hyperlink, 
http://www.gao.gov/products/GAO-14-19]. Washington, D.C.: December 
2, 2013. 

Workforce Investment Act: DOL Should Do More to Improve the Quality of 
Participant Data. [hyperlink, http://www.gao.gov/products/GAO-14-4]. 
Washington, D.C.: December 2, 2013. 

Postsecondary Education: Many States Collect Graduates' Employment 
Information, but Clearer Guidance on Student Privacy Requirements Is 
Needed. [hyperlink, http://www.gao.gov/products/GAO-10-927]. 
Washington, D.C.: September 27, 2010. 

Workforce Investment Act: Employers Found One-Stop Centers Useful in 
Hiring Low-Skilled Workers; Performance Information Could Help Gauge 
Employer Involvement. [hyperlink, 
http://www.gao.gov/products/GAO-07-167]. Washington, D.C.: December 
22, 2006. 

Workforce Investment Act: Labor and States Have Taken Actions to 
Improve Data Quality, but Additional Steps Are Needed. [hyperlink, 
http://www.gao.gov/products/GAO-06-82]. Washington, D.C.: November 14, 
2005. 

Workforce Investment Act: Substantial Funds Are Used for Training, but 
Little Is Known Nationally about Training Outcomes. [hyperlink, 
http://www.gao.gov/products/GAO-05-650]. Washington, D.C.: June 29, 
2005. 

Workforce Investment Act: Labor Should Consider Alternative Approaches 
to Implement New Performance and Reporting Requirements. [hyperlink, 
http://www.gao.gov/products/GAO-05-539]. Washington, D.C.: May 27, 
2005. 

Workforce Investment Act: States and Local Areas Have Developed 
Strategies to Assess Performance, but Labor Could Do More to Help. 
[hyperlink, http://www.gao.gov/products/GAO-04-657]. Washington, D.C.: 
June 1, 2004. 

Workforce Investment Act: Labor Actions Can Help States Improve Quality 
of Performance Outcome Data and Delivery of Youth Services. 
[hyperlink, http://www.gao.gov/products/GAO-04-308]. Washington, D.C.: 
February 23, 2004. 

Workforce Investment Act: Improvements Needed in Performance Measures 
to Provide a More Accurate Picture of WIA's Effectiveness. [hyperlink, 
http://www.gao.gov/products/GAO-02-275]. Washington, D.C.: February 1, 
2002. 

[End of section] 

Footnotes: 

[1] According to DOL, data on training-related employment are 
extremely important for program analysis and evaluation of the value 
of investments in WIA training. U.S. Department of Labor, Training and 
Employment Notice No. 5-13: Reporting Training-Related Employment 
under the Workforce Investment Act (WIA) Programs (September 4, 2013). 

[2] Error rates were not available for training-related employment 
data so we did not consider this factor in state selection. 

[3] WIBs govern local workforce areas. Their responsibilities include 
setting policy for the local workforce area, coordinating workforce 
investment activities with economic development strategies, and 
developing relationships with employers. 

[4] Training and Employment Guidance Letter No. 36-11, Announcement of 
American Job Center Network (Washington, D.C.: June 14, 2012), 
strongly encouraged states and local areas to refer to the one-stop 
career centers as American Job Centers, in order to increase jobseeker 
and employer awareness of available workforce development resources. 

[5] A dislocated worker is an individual who: (1) has been terminated 
or laid off, or has received a notice of termination or layoff from 
employment; (2) is eligible for or has exhausted unemployment 
insurance; or (3) has demonstrated an appropriate attachment to the 
workforce, but is not eligible for unemployment insurance and is 
unlikely to return to a previous industry or occupation; (4) has been 
terminated or laid off or received notification of termination or 
layoff from employment as a result of a permanent closure or 
substantial layoff; (5) is employed at a facility where the employer 
has made the general announcement that the facility will close within 
180 days; (6) was self employed but is unemployed as a result of 
general economic conditions in the community or because of a natural 
disaster; or (7) is a displaced homemaker. See 29 U.S.C. § 2801(9). 

[6] 29 U.S.C. § 2864(d)(4)(E). 

[7] 29 U.S.C. §§ 2864(e)(2) and 2864(e)(3). Supportive services are 
defined at 29 U.S.C. § 2801(46). 

[8] 29 U.S.C. §§ 2831 and 2832. 

[9] 29 U.S.C. § 2832(d). 

[10] Participants must also: 1) select a training program that is 
directly linked to employment opportunities in the local area or in 
another area to which the individual is willing to relocate, and 2) be 
unable to obtain grant assistance from other sources to pay training 
costs. 20 C.F.R. § 663.310. 

[11] See GAO, Workforce Investment Act: Substantial Funds Are Used for 
Training, but Little Is Known Nationally About Training Outcomes. 
[hyperlink, http://www.gao.gov/products/GAO-05-650] (Washington, D.C.: 
June 29, 2005). 

[12] 20 C.F.R. § 663.440(c). 

[13] 29 U.S.C. § 2864(d)(4)(G)(iii). 

[14] Fiscal year 2000 appropriations are presented in nominal dollars 
and are not adjusted for inflation. Thus, in real (inflation-adjusted) 
dollars, appropriated funds for WIA have declined by more than 24 
percent. 

[15] 29 U.S.C. § 2871(b)(2)(A). 

[16] 29 U.S.C. § 2871(b)(2)(B). 

[17] 29 U.S.C. § 2871(b)(3)(A). 

[18] In accordance with WIA, DOL may provide financial awards to 
states for performance that exceeds negotiated levels and assess 
penalties for performance that does not meet negotiated performance 
levels. 20 U.S.C. § 9273 and 29 U.S.C. §2871(g). For all states--those 
with waivers and those without waivers--incentive or sanction 
determinations are based only on the following performance measures: 
entered employment, employment retention, and earnings. See Training 
and Employment Guidance Letter No. 9-07, Revised Incentive and 
Sanction Policy for Workforce Investment Act Title IB Programs 
(Washington, D.C.: October 10, 2007). 

[19] 29 U.S.C. §2871(f)(2). Each state maintains UI wage records to 
support the process of providing unemployment compensation to 
unemployed workers. The UI wage records contain basic wage information 
for about 94 percent of workers. Certain employment categories are 
excluded such as self-employed persons, independent contractors, 
federal employees, and military personnel. If wage information for a 
program participant is not found in the UI wage records, states may 
use supplemental data sources, such as follow-up with the participant 
and employer. 

[20] Common performance measures were developed under an Office of 
Management and Budget initiative which sought to unify definitions for 
performance across programs with similar goals. 

[21] In addition to the 50 states and the District of Columbia, there 
are eight United States territories. As of December 2013, five of 
these eight territories had obtained a common measures waiver. 

[22] States must report data on credentials training participants 
attained either during WIA participation or by the end of the third 
quarter after the quarter of exit from WIA services. States with 
common measures waivers report these data, rather than data on 
training participants who entered employment and attained a 
credential, as was required under the WIA performance measures. U.S. 
Department of Labor, Training and Employment Guidance Letter 17-09, 
Quarterly Submission of Workforce Investment Act Standardized Record 
Data (March 10, 2010). 

[23] U.S. Department of Labor, Training and Employment Guidance Letter 
15-10, Increasing Credential, Degree, and Certificate Attainment by 
Participants of the Public Workforce System (December 15, 2010); 
Debbie Reed et al., An Effectiveness Assessment and Cost-Benefit 
Analysis of Registered Apprenticeship in 10 States, Mathematica Policy 
Research (Oakland, CA: July 25, 2012); W. Norton Grubb, "Learning and 
Earning in the Middle, Part I: National Studies of Pre-baccalaureate 
Education," Economics of Education Review (2002). 

[24] See GAO, Workforce Investment Act: Improvements Needed in 
Performance Measures to Provide a More Accurate Picture of WIA's 
Effectiveness, [hyperlink, http://www.gao.gov/products/GAO-02-275] 
(Washington, D.C.: February 1, 2002). 

[25] DOL defined credential attainment in guidance issued in 2006. In 
2010, DOL issued additional guidance stating that the term 
"credential" refers to an attestation of qualification or competence 
issued to an individual by a third party, such as an educational 
institution, with the relevant authority or assumed competence to 
issue such a credential. U.S. Department of Labor, Training and 
Employment Guidance Letter No. 17-05, Common Measures Policy for the 
Employment and Training Administration's Performance Accountability 
System and Related Performance Issues (February 17, 2006); and U.S. 
Department of Labor, Training and Employment Guidance Letter No. 15-
10, Increasing Credential, Degree, and Certificate Attainment by 
Participants of the Public Workforce System, (December 15, 2010). 

[26] U.S. Department of Labor, Employment and Training Administration, 
Workforce Investment Act Data Reporting and Validation System (DRVS) 
User Handbook for DRVS 7.2 (April 2009). 

[27] 29 U.S.C. §2871(d)(2)(A). DOL requires states to report training-
related employment only on participants that received training and 
entered employment in the first quarter after the quarter of exit from 
WIA services. U.S. Department of Labor, Training and Employment 
Guidance Letter No. 17-09: Quarterly Submission of Workforce 
Investment Act Standardized Record Data (WIASRD) (March 10, 2010). 

[28] U.S. Department of Labor, Training and Employment Guidance Letter 
No. 17-09, Quarterly Submission of Workforce Investment Act 
Standardized Record Data (WIASRD)(March 10, 2010). 

[29] U.S. Department of Labor Office of Inspector General Audit Report 
No. 03-11-003-03-390, Additional Information Needed to Measure the 
Effectiveness and Return on Investment of Training Services Funded 
Under the Adult and Dislocated Worker Programs (September 30, 2011). 

[30] GAO, Performance Plans: Selected Approaches for Verification and 
Validation of Agency Performance Information, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-99-139] (Washington, D.C.: July 
30, 1999). 

[31] U.S. Department of Labor, Office of Performance and Technology, 
Employment and Training Administration, PY 2008 WIA Trends Over Time 
(January 19, 2010); and PY 2011 WIA Trends Over Time (December 13, 
2012). 

[32] Throughout the report, "received training" is used to describe 
participants who entered training, but may or may not have completed 
it. 

[33] A general slowdown in economic activity, a downturn in the 
business cycle, and a reduction in the amount of goods and services 
produced and sold are all characteristics of a recession. The most 
recent recession began in December 2007 and ended in June 2009. U.S. 
Bureau of Labor Statistics, The Recession of 2007-2009 (February 2012). 

[34] Pub. L. No. 111-5, 123 Stat, 115, 172-173. 

[35] For WIA Adult and Dislocated Worker Programs, states report 
aggregate counts of all participants, including those whose services 
were funded with regular WIA formula funds and Recovery Act funds. 

[36] DOL defined the WIA performance measure on credential attainment 
to assess training participants who secured employment and received a 
credential. U.S. Department of Labor, Training and Employment Guidance 
Letter No. 7-99, Core and Customer Satisfaction Performance Measures 
for the Workforce Investment System (March 3, 2000). 

[37] Occupational licenses are required by government entities, 
typically state regulatory bodies, before an individual is allowed to 
be employed in and practice a trade, profession, or other occupation. 

[38] GAO, Workforce Investment Act: DOL Should Do More to Improve the 
Quality of its Participant Data. [hyperlink, 
http://www.gao.gov/products/GAO-14-4] (Washington, D.C.: December 2, 
2013). 

[39] U.S. Department of Labor, Office of Inspector General, Additional 
Information Needed to Measure the Effectiveness and Return on 
Investment of Training Services Funded Under WIA Adult and Dislocated 
Worker Programs, Report Number 03-11-003-03-390 (September 30, 2011). 

[40] 20 U.S.C. § 1232g. FERPA includes requirements related to the use 
and disclosure of education records on individual students and 
provides that, generally, a student must provide written consent 
before an educational agency or institution may disclose personally 
identifiable information from that student's education records. 
Directory information, including major field of study and the degrees 
received, may be released without prior consent of a student if 
certain conditions are met: an educational agency or institution must 
give public notice of: (1) the types of personally identifiable 
information the educational agency or institution has designated as 
directory information, (2) the right of the student to refuse to let 
the educational agency or institution designate those types of 
information as directory information, and (3) the period of time 
within which a student can notify the educational agency or 
institution of that refusal. An educational agency or institution may 
disclose directory information about former students unless, while in 
attendance, the student opted out of the disclosure of directory 
information and has not rescinded that request. 34 C.F. R. § 99.37. 

[41] Officials from the Department of Education said that FERPA 
generally does not apply to third-party organizations that grant 
credentials to an individual who takes a test through a third-party 
organization and applies directly to that organization to receive a 
credential. However, if such a third-party organization were to 
provide an educational agency or institution covered by FERPA with a 
copy of the test results or credential that directly relate to a 
student attending their program and that information were maintained 
by the educational agency or institution, then that information would 
become subject to FERPA. 

[42] See GAO, Workforce Investment Act: Improvements Needed in 
Performance Measures to Provide a More Accurate Picture of WIA's 
Effectiveness, [hyperlink, http://www.gao.gov/products/GAO-02-275] 
(Washington, D.C.: February 1, 2002). 

[43] U.S. Department of Labor, Training and Employment Guidance Letter 
17-05, Common Measures Policy for the Employment and Training 
Administration's (ETA) Performance Accountability System and Related 
Performance Issues (February 17, 2006). 

[44] U.S. Department of Labor, Training and Employment Guidance Letter 
15-10, Increasing Credential, Degree, and Certificate Attainment by 
Participants of the Public Workforce System (December 15, 2010). 

[45] The fiscal year 2013 goal included a broader array of workforce 
programs than the prior goal, including National Emergency Grants, 
Trade Adjustment Assistance, Green Jobs Innovation Fund, and Job 
Corps, among others. 

[46] Washington State Employment Security Department, Certificates and 
Credentials Under Common Measures, Policy Number 3690, Revision 1 
(September 23, 2011). 

[47] The National Student Clearinghouse offers access to nationwide 
coverage of enrollment and degree records --encompassing over 98 
percent of all students in public and private U.S. institutions. None 
of our other selected states mentioned using this source for 
collecting data on credential attainment. 

[48] WIA defines "on-the-job training" as training by an employer that 
is provided to a paid participant while engaged in productive work in 
a job that: a) provides knowledge or skills essential to the full and 
adequate performance of the job; b) reimburses the employer up to 50 
percent of the participant's wage rate for the costs associated with 
training and related supervision; and c) is limited in duration as 
appropriate to the particular occupation, taking into account the 
content of the training, the prior work experience of the participant, 
and the service strategy of the participant, as appropriate. 29 U.S.C. 
§ 2801(31). 

[49] GAO, Workforce Investment Act: Local Areas Face Challenges 
Helping Employers Fill Some Types of Skilled Jobs. [hyperlink, 
http://www.gao.gov/products/GAO-14-19] (Washington, D.C.: December 2, 
2013). 

[50] Work readiness certificates generally refer to awards that 
demonstrate achievement of a certain level of workplace employability 
skills. One common program is ACT's National Career Readiness 
Certificate, which can be earned through the Work Keys program. 

[51] DOL grants waivers that can exempt a state from reporting 
credential attainment outcomes for participants enrolled in OJT. 

[52] DOL does not require documentation on training-related employment 
because it is not a WIA performance measure. 

[53] U.S. Department of Labor, Training and Employment Guidance Letter 
No. 17-09: Quarterly Submission of Workforce Investment Act 
Standardized Record Data (WIASRD) (March 10, 2010). 

[54] UI wage data are collected quarterly, meaning that participants 
who find employment will not show up in the data until several months 
after they are hired. As a result, case managers must wait several 
months to use these data. 

[55] DOL's O*NET database provides detailed information about many 
occupations. The database is online, available to the public at no 
cost, and is continually updated by surveying a broad range of workers 
from each occupation. 

[56] U.S. Department of Labor, Office of Inspector General, Additional 
Information Needed to Measure the Effectiveness and Return on 
Investment of Training Services Funded Under the WIA Adult and 
Dislocated Worker Programs, Report Number 03-11-003-03-390 (September 
30, 2011). 

[57] U.S. Department of Labor, Training and Employment Notice No. 5-
13: Reporting Training-Related Employment under the Workforce 
Investment Act (WIA) Programs (September 4, 2013). 

[58] DOL also established a new WIASRD response option to isolate 
situations in which case managers do not know if participants' 
employment was training-related. 

[59] GAO, Performance Plans: Selected Approaches for Verification and 
Validation of Agency Performance Information, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-99-139] (Washington, D.C.: July 
30, 1999). 

[60] According to DOL officials, states must report on the number of 
participants who find training-related employment within three 
quarters of their exit from WIA services. 

[61] Program year 2011 runs from July 1, 2011 through June 30, 2012. 
WIASRD is a national database of individual records with information 
on the characteristics, activities, and outcomes for all enrolled 
participants who received services or benefits under WIA. Participant 
data are typically collected by frontline staff at local American Job 
Centers and entered into a state or local data system. After the state 
receives data from local areas, it compiles and formats the 
information and submits it electronically to DOL. 

[62] Error rates were not available for data on training-related 
employment, so we did not consider these data in state selection. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]