On-the-Job Training: Federal Highway Administration Needs to Strengthen Program Assessment
Highlights
The Federal Highway Administration's (FHWA) on-the-job training program-- a relatively small part of the federal aid highway program--requires states to implement job training programs to provide traditionally underrepresented groups with opportunities in highway construction. To increase the effectiveness of state job training programs, FHWA grants up to $10 million annually for supportive services, such as job placement assistance. This report examines the extent to which (1) FHWA's job training program enhances training and career opportunities for these groups, (2) FHWA oversees the job training programs, and (3) supportive services provide assistance to these programs. To address these topics GAO reviewed federal legislation, good management practices identified in prior GAO reports, FHWA documents, and proposals and reports submitted by states. GAO conducted an in-depth examination of these efforts in four states, and interviewed a cross-section of FHWA staff, state officials, and industry groups.
It is unclear the extent to which FHWA's on-the-job training program enables women, minorities, and economically disadvantaged individuals to reach journeylevel status in the highway construction trades, although stakeholders believe it can create some opportunities. FHWA's decentralized management of the program--in which state transportation agencies and FHWA's division offices are generally responsible for program implementation--has led to a wide range of practices. As a result, the types of training opportunities created by the program vary from state to state in terms of, for example, the length of training and the entities involved in providing training. In addition, the extent that state programs focus on creating training opportunities for traditionally underrepresented groups differs. The limited amount of useable information available on program results varies among states. As a result, FHWA does not know how well the program is doing, and GAO could not accurately determine how many trainees participate in the program or the demographics of those trainees; however, GAO estimates that several thousand likely participate in any one year. FHWA's oversight approach does little to assess program results. FHWA lacks clear criteria that articulate what states are supposed to accomplish through their job training programs. While some broad program expectations are stipulated in guidance and regulations, FHWA acknowledges some of these are outdated. Furthermore, FHWA's oversight approach does not determine the overall effectiveness of state programs or measure state progress. For example, although state transportation agencies are required to submit achievement information on an annual basis to FHWA division offices, states submitted this information using a wide range of different output terms and different demographic and trade classification categories. GAO has reported that program criteria are key aspects of results-oriented performance management. Through a separate program, FHWA provides funding for a variety of activities intended to increase the overall effectiveness of the on-the-job training program, but its overall stewardship of the program is limited. FHWA's supportive services program provides grants for locally tailored initiatives, such as skills training, child care, and career awareness events, that directly and indirectly link to job training programs. However, there is insufficient data to determine how effective these efforts have been in enhancing job training opportunities. Although FHWA has articulated the types of data states should collect and report, the agency does not know, and GAO could not determine, the number of participants in the supportive services program or its effect, in part because grantees do not always provide information about their program results. However, GAO estimated that there are about 10,000 people participating in any one year. Furthermore, past performance information is not required of applicants or scored during funding reviews. Given that many grantees are funded repeatedly, good management practices suggest that using past performance information can inform and improve recipient selection approaches. Program results are important for making budgetary and programmatic decisions. Without insight into program activities, FHWA cannot ensure that funding is used effectively. GAO recommends that FHWA (1) strengthen on-the-job training program criteria, (2) create and implement an oversight approach for its job training program, and (3) evaluate the extent to which supportive services programs have met their goals and use this information to inform future funding decisions. The agency generally agreed with these recommendations and provided technical comments, which were incorporated as appropriate.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Transportation | To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should direct the FHWA Administrator to strengthen criteria--through regulations, guidelines, or other mechanisms--so that states have a clear understanding of how the on-the-job training program should be implemented and the results state programs are intended to accomplish. |
Closed – Implemented
The Federal Highway Administration's (FHWA) on-the-job training program requires states to implement job training programs to provide traditionally underrepresented groups with opportunities in highway construction. In 2011, GAO reported that FHWA lacked clear criteria that articulate what states were supposed to accomplish through their job training programs. While FHWA established regulations and broad guidelines for the program that provide some assistance to state transportation agencies, the guidelines were vague and it was unclear what states should accomplish. FHWA acknowledged some program regulations were outdated and had not been updated to reflect changes in program implementation. While these regulations and guidelines established general criteria for states, they did not establish clear criteria for what states must do, what is optional, and the degree to which states can adapt their programs to their own circumstances. FHWA left the interpretation of these regulations and guidelines, and subsequent program implementation, largely to state transportation agencies and FHWA division offices. Because FHWA had not established clear criteria, state transportation agencies relied on their own judgment in implementing their programs and the types of training opportunities created by the program varied across states. Previously, GAO has reported that program criteria are key aspects of results-oriented performance management. The lack of clear criteria can make program implementation more difficult for states and prevents FHWA from having a basis to judge the effectiveness of varying state programs in advancing women, minorities, and economically disadvantaged individuals in highway construction trades. Accordingly, GAO recommended that FHWA strengthen federal OJT program criteria--through regulations, guidelines, or other mechanisms--so that states have a clear understanding of how their OJT programs should be implemented and what they should accomplish. In 2017, GAO confirmed that FHWA took steps to strengthen its OJT program's criteria. First, in 2014 FHWA issued a new program handbook for a variety of its civil rights programs that includes OJT program guidance. This handbook states that the intent of the OJT program is to move women, minorities, and disadvantaged individuals into journey-level positions in highway construction and address historical underrepresentation. The handbook also indicates different program approaches states may use in their programs that are acceptable to FHWA. In 2016, FHWA also took less formal steps to clarify program requirements and strengthen and clarify program criteria, giving states a better understanding of the program's intent. These steps included holding webinars and conference presentations for numerous state transportation officials about the program, creating new mechanisms to share and promote noteworthy practices, and establishing the Center for Transportation Workforce Development to advance innovative practices in support of the OJT program. As a direct result, FHWA and states are better able to implement their OJT programs in accordance with federal program criteria and, ultimately, advance the broader goal of bringing underrepresented individuals into the highway construction workforce.
|
| Department of Transportation | To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should direct the FHWA Administrator to create and implement an oversight mechanism that (1) holds states accountable for meeting federal training criteria and (2) clearly stipulates how FHWA will assess state program effectiveness, including what type of program achievement data states are to submit and how such data will be used. This oversight mechanism should include assessing the effectiveness of its division offices in overseeing state activities. |
Closed – Implemented
The Federal Highway Administration's (FHWA) on-the-job (OJT) training program requires state transportation agencies to implement OJT training programs that provide work opportunities in highway construction to traditionally underrepresented groups. In 2011, GAO reported FHWA's oversight approach did little to assess its OJT training program nationally and did not determined whether states' efforts had led to increased opportunities for workers traditionally underrepresented in the highway construction workforce. FHWA lacked clear criteria that articulated what states were supposed to accomplish through their job training programs. While FHWA established regulations and broad guidelines for the program, the guidelines were vague and outdated according to the agency. Furthermore, FHWA's oversight approach did not determine the overall effectiveness of state programs or measure state progress. For example, although state transportation agencies were required to submit achievement information on an annual basis to FHWA division offices, states submitted this information using a wide range of different output terms and different demographic and trade classification categories. Previously, GAO reported that program criteria are key aspects of results-oriented performance management. In the absence of clear criteria on how division offices should oversee state programs and how to hold state transportation agencies accountable, FHWA was unable to fully assess whether division offices were effective in ensuring that states meet program requirements. Therefore, GAO recommended that FHWA create and implement an oversight mechanism that (1) holds states accountable for meeting federal training criteria and (2) clearly stipulates how FHWA will assess state program effectiveness, including what type of program achievement data states are to submit, how such data will be used, and how FHWA's division offices' oversight will be assessed. In response to this recommendation, FHWA has taken actions in recent years to create and implement mechanisms to hold states accountable. In 2022, GAO confirmed that FHWA implemented Civil Rights Connect, a web portal for states to report program data and results to FHWA. FHWA also established a new annual reporting tool to oversee and track OJT training programs that states are required to submit through Civil Rights Connect. This tool and associated instructions clearly stipulated how states may set OJT goals and what information, such as demographic and trade information, states should report to FHWA each year. FHWA also instructed states to report the reasons why, if applicable, it did not meet OJT goals and the steps planned for the next year to meet goals. FHWA's roll out of these tools included a webinar for FHWA division staff and state officials on their oversight and reporting responsibilities. According to FHWA, the information is used by FHWA to identify national trends in OJT training program implementation. Finally, FHWA's national OJT report is using this new information to summarize states' programs and division activities to hold states accountable. Moreover, the report describes gaps in states' reporting and ongoing FHWA efforts to address these gaps, providing a way to assess state efforts. As a result of these FHWA actions, FHWA is better (1) able to ensure that states are being held accountable to the program's goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce and (2) assess the oversight activities of division offices in order to monitor states' progress in meeting program requirements.
|
| Department of Transportation | To establish accountability for meeting the programs' goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, the Secretary of Transportation should, for the supportive services program, develop an approach to (1) evaluate the extent to which grantees have met their proposed annual goals and (2) integrate the results of this evaluation into FHWA's funding decisions for supportive services programs. |
Closed – Implemented
n 2011, GAO reported that FHWA grants about $10 million each year for supportive services that directly and indirectly support job training activities in highway construction, such as skills training, child care, and career awareness events. However, even after considering the small size of the supportive services program, GAO found that FHWA's stewardship is limited. FHWA has not used any of the information available to it to determine the overall effectiveness of its supportive services activities. FHWA does not track how many of the state grantees have achieved their goals in directly supporting on-the-job training. Although the number of participants in the supportive services program could not be determined, GAO estimated that there were about 10,000 people participating in any one year. Furthermore, past performance information is not required of applicants or scored during funding reviews. Given that many grantees are funded repeatedly, good management practices suggest that using past performance information can inform and improve recipient selection approaches. Moreover, FHWA had not established how state-reported data can be used to evaluate whether grantees have met FHWA's overall program objectives. Therefore, GAO recommended that FHWA evaluate the extent to which supportive services programs have met their goals and use this information to inform future funding decisions. In 2012, after GAO's report was issued, FHWA changed the supportive services program from a discretionary grant program to a formula-based program under which FHWA allocates program funds to states through a formula. State transportation agencies are then responsible for making award decisions to service providers and monitoring their performance. Nevertheless, FHWA remains the ultimate steward of these federal dollars. Accordingly, FHWA should conduct oversight to ensure program goals are met and that performance information is used by states in making funding decisions. In implementing the new process, FHWA took several steps to address GAO's concerns that led to its recommendation. First, in 2013, FHWA developed and implemented a performance information dashboard to collect and monitor the results of the supportive services program. Second, in 2014, FHWA distributed program guidance that clarifies the roles and responsibilities of state transportation agencies and FHWA in administering the supportive services program under the new formula-based program structure. The guidance included information about how states should monitor performance and evaluate program results and that annual program accomplishment reports by the state should be tied to planned program objectives and goals. Further, FHWA strengthened the role of its division offices, which are located in each state, in overseeing the supportive services program in their state, including reviewing proposed supportive services projects to ensure they have measurable goals and subsequently monitoring performance. In 2014, FHWA also held a webinar for FHWA division offices and state transportation agencies on collecting and evaluate grantee performance and to inform funding decisions. Finally, in 2015, GAO confirmed that FHWA divisions are monitoring state decision making about how to use OJT/SS grants and that states are incorporating performance information into these decisions. Collectively, FHWA has taken sufficient steps to address GAO's concerns and the intent of its recommendation. Furthermore, these steps should lay the ground work for evaluation of supportive services results that will inform funding decisions.
|