On-the-Job Training:

Federal Highway Administration Needs to Strengthen Program Assessment

GAO-11-703: Published: Sep 7, 2011. Publicly Released: Oct 7, 2011.

Additional Materials:

Contact:

David J. Wise
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Federal Highway Administration's (FHWA) on-the-job training program-- a relatively small part of the federal aid highway program--requires states to implement job training programs to provide traditionally underrepresented groups with opportunities in highway construction. To increase the effectiveness of state job training programs, FHWA grants up to $10 million annually for supportive services, such as job placement assistance. This report examines the extent to which (1) FHWA's job training program enhances training and career opportunities for these groups, (2) FHWA oversees the job training programs, and (3) supportive services provide assistance to these programs. To address these topics GAO reviewed federal legislation, good management practices identified in prior GAO reports, FHWA documents, and proposals and reports submitted by states. GAO conducted an in-depth examination of these efforts in four states, and interviewed a cross-section of FHWA staff, state officials, and industry groups.

It is unclear the extent to which FHWA's on-the-job training program enables women, minorities, and economically disadvantaged individuals to reach journeylevel status in the highway construction trades, although stakeholders believe it can create some opportunities. FHWA's decentralized management of the program--in which state transportation agencies and FHWA's division offices are generally responsible for program implementation--has led to a wide range of practices. As a result, the types of training opportunities created by the program vary from state to state in terms of, for example, the length of training and the entities involved in providing training. In addition, the extent that state programs focus on creating training opportunities for traditionally underrepresented groups differs. The limited amount of useable information available on program results varies among states. As a result, FHWA does not know how well the program is doing, and GAO could not accurately determine how many trainees participate in the program or the demographics of those trainees; however, GAO estimates that several thousand likely participate in any one year. FHWA's oversight approach does little to assess program results. FHWA lacks clear criteria that articulate what states are supposed to accomplish through their job training programs. While some broad program expectations are stipulated in guidance and regulations, FHWA acknowledges some of these are outdated. Furthermore, FHWA's oversight approach does not determine the overall effectiveness of state programs or measure state progress. For example, although state transportation agencies are required to submit achievement information on an annual basis to FHWA division offices, states submitted this information using a wide range of different output terms and different demographic and trade classification categories. GAO has reported that program criteria are key aspects of results-oriented performance management. Through a separate program, FHWA provides funding for a variety of activities intended to increase the overall effectiveness of the on-the-job training program, but its overall stewardship of the program is limited. FHWA's supportive services program provides grants for locally tailored initiatives, such as skills training, child care, and career awareness events, that directly and indirectly link to job training programs. However, there is insufficient data to determine how effective these efforts have been in enhancing job training opportunities. Although FHWA has articulated the types of data states should collect and report, the agency does not know, and GAO could not determine, the number of participants in the supportive services program or its effect, in part because grantees do not always provide information about their program results. However, GAO estimated that there are about 10,000 people participating in any one year. Furthermore, past performance information is not required of applicants or scored during funding reviews. Given that many grantees are funded repeatedly, good management practices suggest that using past performance information can inform and improve recipient selection approaches. Program results are important for making budgetary and programmatic decisions. Without insight into program activities, FHWA cannot ensure that funding is used effectively. GAO recommends that FHWA (1) strengthen on-the-job training program criteria, (2) create and implement an oversight approach for its job training program, and (3) evaluate the extent to which supportive services programs have met their goals and use this information to inform future funding decisions. The agency generally agreed with these recommendations and provided technical comments, which were incorporated as appropriate.

Recommendations for Executive Action

  1. Status: Open

    Comments: FHWA has taken some initial steps that may address this recommendation. GAO is awaiting further information on the status of these actions.

    Recommendation: To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should direct the FHWA Administrator to strengthen criteria--through regulations, guidelines, or other mechanisms--so that states have a clear understanding of how the on-the-job training program should be implemented and the results state programs are intended to accomplish.

    Agency Affected: Department of Transportation

  2. Status: Open

    Comments: FHWA has taken some initial steps that may address this recommendation. GAO is awaiting further information on the status of these actions.

    Recommendation: To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should direct the FHWA Administrator to create and implement an oversight mechanism that (1) holds states accountable for meeting federal training criteria and (2) clearly stipulates how FHWA will assess state program effectiveness, including what type of program achievement data states are to submit and how such data will be used. This oversight mechanism should include assessing the effectiveness of its division offices in overseeing state activities.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: n 2011, GAO reported that FHWA grants about $10 million each year for supportive services that directly and indirectly support job training activities in highway construction, such as skills training, child care, and career awareness events. However, even after considering the small size of the supportive services program, GAO found that FHWA's stewardship is limited. FHWA has not used any of the information available to it to determine the overall effectiveness of its supportive services activities. FHWA does not track how many of the state grantees have achieved their goals in directly supporting on-the-job training. Although the number of participants in the supportive services program could not be determined, GAO estimated that there were about 10,000 people participating in any one year. Furthermore, past performance information is not required of applicants or scored during funding reviews. Given that many grantees are funded repeatedly, good management practices suggest that using past performance information can inform and improve recipient selection approaches. Moreover, FHWA had not established how state-reported data can be used to evaluate whether grantees have met FHWA's overall program objectives. Therefore, GAO recommended that FHWA evaluate the extent to which supportive services programs have met their goals and use this information to inform future funding decisions. In 2012, after GAO's report was issued, FHWA changed the supportive services program from a discretionary grant program to a formula-based program under which FHWA allocates program funds to states through a formula. State transportation agencies are then responsible for making award decisions to service providers and monitoring their performance. Nevertheless, FHWA remains the ultimate steward of these federal dollars. Accordingly, FHWA should conduct oversight to ensure program goals are met and that performance information is used by states in making funding decisions. In implementing the new process, FHWA took several steps to address GAO's concerns that led to its recommendation. First, in 2013, FHWA developed and implemented a performance information ?dashboard? to collect and monitor the results of the supportive services program. Second, in 2014, FHWA distributed program guidance that clarifies the roles and responsibilities of state transportation agencies and FHWA in administering the supportive services program under the new formula-based program structure. The guidance included information about how states should monitor performance and evaluate program results and that annual program accomplishment reports by the state should be tied to planned program objectives and goals. Further, FHWA strengthened the role of its division offices, which are located in each state, in overseeing the supportive services program in their state, including reviewing proposed supportive services projects to ensure they have measurable goals and subsequently monitoring performance. In 2014, FHWA also held a webinar for FHWA division offices and state transportation agencies on collecting and evaluate grantee performance and to inform funding decisions. Finally, in 2015, GAO confirmed that FHWA divisions are monitoring state decision making about how to use OJT/SS grants and that states are incorporating performance information into these decisions. Collectively, FHWA has taken sufficient steps to address GAO's concerns and the intent of its recommendation. Furthermore, these steps should lay the ground work for evaluation of supportive services results that will inform funding decisions.

    Recommendation: To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should, for the supportive services program, develop an approach to (1) evaluate the extent to which grantees have met their proposed annual goals and (2) integrate the results of this evaluation into FHWA's funding decisions for supportive services programs.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Dec 7, 2016

Dec 2, 2016

Nov 25, 2016

Nov 22, 2016

Nov 17, 2016

Nov 15, 2016

Nov 9, 2016

Oct 31, 2016

Looking for more? Browse all our products here