This is the accessible text file for GAO report number GAO-06-156 entitled 'Large Truck Safety: Federal Enforcement Efforts Have Been Stronger Since 2000, but Oversight of State Grants Needs Improvement' which was released on December 15, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: December 2005: Large Truck Safety: Federal Enforcement Efforts Have Been Stronger Since 2000, but Oversight of State Grants Needs Improvement: [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-156] GAO Highlights: Highlights of GAO-06-156, a report to congressional committees: Why GAO Did This Study: About 5,000 people die and more than 120,000 are injured each year from crashes involving large trucks. The Federal Motor Carrier Safety Administration (FMCSA) has several enforcement programs to improve truck safety and funds similar enforcement programs in states through its Motor Carrier Safety Assistance Program (MCSAP). Following concern by Congress and others in 1999 that FMCSA’s enforcement approach was ineffective, the agency committed to take stronger actions. This study reports on how FMCSA’s enforcement approach has changed, how it makes decisions about its enforcement approach, and how it ensures that its grants to states contribute to the agency’s mission of saving lives. What GAO Found: FMCSA has made considerable strides in strengthening its enforcement programs and actions. For example, it almost doubled the number of on- site safety reviews (called compliance reviews) at carriers’ bases of operations, from approximately 6,400 in 1998 to 11,300 in 2004. Further, it has increased the average civil penalty by about 75 percent (from $820 to $1,400) over the same period. FMCSA generally maintained its firmer approach to enforcement at a time when it took on the additional responsibilities of conducting homeland security-related reviews of hazardous materials carriers and safety reviews of new carriers. To a large extent, FMCSA follows key effective management practices in making decisions about its enforcement approach. For example, its enforcement approach addresses major risk areas that contribute to (or cause) crashes, and targets its enforcement resources at the motor carriers with the greatest crash risk. FMCSA also has a broad range of enforcement goals and performance measures that it uses to provide direction to—and track the performance of—its enforcement programs. Furthermore, FMCSA is working to obtain additional information on crash risk factors and on the costs and effectiveness of its enforcement programs, as well as alternative approaches that it needs to further refine and set priorities for its programs. However, because FMCSA does not measure the effect that one of its key enforcement tools—civil penalties—has on carriers’ compliance with safety regulations, it lacks the information needed to make sound decisions about any changes to its use of civil penalties. MCSAP is designed to improve safety by employing a performance-based approach; however, FMCSA’s oversight for these grants is inadequate. In reviewing the 61 program goals set by the seven states that received the largest MCSAP grants, we could not determine whether states substantially met almost two-thirds of these goals due to missing performance information, among other reasons. Further, although FMCSA requires that its various offices periodically review grant activities for adequacy of oversight, few of these reviews are being completed. For example, in the past 3 years, FMCSA’s service centers have assessed only 15 of the agency’s 52 field division offices (29 percent). FMCSA did not conduct these reviews for various reasons, including a curbed oversight role for service centers and markedly reduced headquarters staffing for MCSAP. FMCSA’s progress on initiatives to improve truck safety enforcement: [See PDF for image] [End of figure] What GAO Recommends: GAO makes several recommendations to improve FMCSA’s ability to determine the effectiveness of its enforcement approach and strengthen the agency’s oversight of MCSAP, such as measuring the effectiveness of its civil penalties and ensuring that existing planning and oversight mechanisms are carried out. GAO provided a draft of this report to the Department of Transportation for its review and comment. The department generally agreed with the report’s findings and agreed to consider our recommendations. www.gao.gov/cgi-bin/getrpt?GAO-06-156. To view the full product, including the scope and methodology, click on the link above. For more information, contact Katherine Siggerud at (202) 512-2834 or siggerudk@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: FMCSA's Enforcement Programs and Actions Have Been Stronger Since 2000: FMCSA Has a Well-Defined Enforcement Approach and Has Efforts Underway to Help It Refine and Set Priorities for Its Enforcement Programs: MCSAP Is Designed to Improve Safety but Program Oversight Is Inadequate: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendixes: Appendix I: Scope and Methodology: Appendix II: GAO Contact and Staff Acknowledgements: Tables: Table 1: FMCSA's Enforcement Programs: Table 2: FMCSA's Enforcement Actions: Table 3: How FMCSA's Enforcement Approach Addresses Many of the Major Factors that Cause or Contribute to Truck Crashes: Table 4: Examples of FMCSA's Enforcement Program Goals and Performance Measures: Figures: Figure 1: Growth in the Number of Interstate Motor Carriers, 1995 through 2004 Figure 2: Fatality Rates Involving Large Truck and Passenger Vehicle Crashes, 1995 through 2004 Figure 3: Number of Compliance Reviews Conducted and Percentage of the Motor Carrier Industry Receiving Compliance Reviews, 1995 through 2004 Figure 4: Roadside Inspections Conducted and Planned, 1996 through 2006 Figure 5: Number of Serious Violations Found and Rate of Enforcement, 1995 through 2004 Figure 6: Number of Carriers Prohibited from Operating in Interstate Commerce by FMCSA, 1995 through 2004 Figure 7: Average Civil Penalty Assessed per Violation, by Type of Violation, Fiscal Years 1995 through 2004 (In 2004 Dollars) Figure 8: Final Assessment as Compared to Initial Assessment, Fiscal Years 1995 through 2004 Figure 9: Logic Model of How FMCSA's Enforcement Approach Contributes to Truck Safety: Abbreviations: FMCSA: Federal Motor Carrier Safety Administration: MCSAP: Motor Carrier Safety Assistance Program: PRISM: Performance Registration and Information System Management: Letter December 15, 2005: The Honorable Christopher Bond: Chairman: The Honorable Patty Murray: Ranking Minority Member: Subcommittee on Transportation, Treasury, the Judiciary, Housing and Urban Development, and Related Agencies: Committee on Appropriations: United States Senate: The Honorable Joe Knollenberg: Chairman: The Honorable John W. Olver: Ranking Minority Member: Subcommittee on Transportation, Treasury, and Housing and Urban Development, the Judiciary, the District of Columbia, and Independent Agencies: Committee on Appropriations: House of Representatives: About 5,000 people die each year as a result of crashes involving large trucks,[Footnote 1] and over 120,000 more are injured. Compared to a crash involving only cars, a crash involving a car and a truck is more likely to result in a fatality because of the difference in weight between the two vehicles. In this respect, although large trucks are involved in only 4 percent of all accidents, they are involved in 12 percent of all fatalities from vehicle crashes. These crashes may result from errors by truck and passenger vehicle drivers, vehicle condition, and other factors. The Federal Motor Carrier Safety Administration (FMCSA) within the U.S. Department of Transportation shoulders the primary federal responsibility for reducing crashes, injuries, and fatalities involving large trucks.[Footnote 2] FMCSA's primary means of preventing these crashes is to develop and enforce regulations to help ensure that drivers and motor carriers are operating in a safe manner. FMCSA uses several enforcement activities to improve truck safety, such as conducting inspections of motor carriers' operations at their places of business--and of drivers and vehicles at the roadside--to ensure compliance with safety regulations. FMCSA also funds and oversees similar enforcement activities at the state level through its Motor Carrier Safety Assistance Program (MCSAP). MCSAP was appropriated $188 million, or about 38 percent, of FMCSA's $501 million appropriation for fiscal year 2006. FMCSA was created in 2000 in response to concerns raised by Congress, the Department of Transportation's Inspector General, and us. One of the Inspector General's concerns was over the lax enforcement of safety regulations by FMCSA's predecessor, the Office of Motor Carriers, within the Federal Highway Administration, in the late 1990s.[Footnote 3] FMCSA publicly committed to increasing both the amount of effort devoted to enforcement against carriers and the severity of action when safety problems are found. In addition, in 1999 we reported that FMCSA did not have sufficient information about the factors that contribute to truck crashes to help the agency set priorities for its activities. In a similar vein, in 2000, we found that the agency's ability to set priorities for its safety activities was limited, in part, by the agency's lack of knowledge about the safety impact of its activities. In fiscal year 2003, the Office of Management and Budget identified challenges to program accountability within MCSAP. The Senate report for the Department of Transportation's fiscal year 2005 appropriation directed us to examine the effectiveness of FMCSA's truck safety enforcement activities. Accordingly, and as discussed with Senate Appropriations Committee staff members, this report focuses on: * how FMCSA's safety enforcement approach, programs, and actions have changed since 2000; * the extent to which FMCSA follows key effective management practices in making decisions about its safety enforcement approach; and: * the extent to which FMCSA ensures that its grants to states contribute to the agency's mission of saving lives and reducing injuries by preventing truck crashes. To examine how FMCSA's enforcement approaches, programs, and actions have changed since 2000, we reviewed legislation and FMCSA regulations, program guidance, and plans. We also reviewed congressional reports related to the creation of FMCSA, and assessments of the agency's enforcement efforts by the Department of Transportation's Office of Inspector General and by us. We analyzed data from FMCSA on its enforcement activities, such as the number of reviews of carrier compliance with safety regulations (referred to as "compliance reviews"). We also analyzed FMCSA data on enforcement actions, such as civil penalties assessed from fiscal year 1995 (before FMCSA committed to be more aggressive) through 2004 (the latest full year for which data are available). Finally, we discussed these issues with industry and safety advocates. To determine the extent to which FMCSA follows key effective management practices in making decisions about its enforcement approach, we determined the extent to which the agency incorporated several key elements that are important for effective program management: (1) use of program goals, including goals that describe the intended outcomes of programs; (2) a well-defined approach for achieving goals; and (3) performance measures that demonstrate contributions to program goals. We identified elements of effective program management by reviewing our reports on this topic, Office of Management and Budget guidance, and the Government Performance and Results Act of 1993. We reviewed FMCSA documents about the agency's activities and plans related to each of these elements, and we also interviewed agency officials. To assess the extent to which FMCSA ensures that its MCSAP grants contribute to the agency's mission, we reviewed regulations and FMCSA guidance relating to the design of the grant program. We also discussed with FMCSA officials how accountability is built into the grant program. We assessed FMCSA's planning and oversight of seven state MCSAP grantees in fiscal year 2004 (the latest full year for which information was available), including whether FMCSA obtained sufficient information to be able to determine whether grantees substantially met their objectives. We then discussed our assessment with FMCSA officials responsible for grant activities in those states. The seven states represent about 27 percent of MCSAP grant funding in fiscal year 2004. Because we chose these states judgmentally (representing the largest grantees), we cannot project our findings nationwide. Reviewing a larger number of grantees would not have been practical due to resource constraints. As part of our review, we assessed internal controls and the reliability of FMCSA's data on its program activities and enforcement actions that were pertinent to this effort. We determined that the data elements were sufficiently reliable for our purposes. We conducted our work from October 2004 to December 2005 in accordance with generally accepted government auditing standards. (See app. I for additional information on our scope and methodology.) Results in Brief: FMCSA has increased both its enforcement activities and actions in response to criticism in 1999 that it was ineffective at improving safety. The agency more than doubled the number of carrier reviews from about 6,400 in 1998 to about 13,200 in 2000, and increased the average civil penalty per violation by more than 80 percent from about $820 to about $1,500 over the same time period (in constant 2004 dollars). In the 5 years since FMCSA implemented these changes, some of FMCSA's enforcement activities and actions have remained relatively steady, while others have fallen slightly, in part because FMCSA's statutory responsibilities have expanded to include conducting homeland security- related reviews of hazardous materials carriers and educating new carriers about their responsibilities under the safety regulations. FMCSA has also begun to reduce civil penalties for first-time offenders in cases where the carrier agrees to come into compliance and make additional safety-related improvements. At the same time, FMCSA officials told us that the agency is committed to applying strict enforcement to egregious offenders. To a large extent, FMCSA follows key effective management practices in making decisions about its enforcement approach. For example: * FMCSA has a well-defined enforcement approach that addresses major risk areas that contribute to (or cause) crashes, such as motor carrier operations and driver behavior, and targets resources at the motor carriers that FMCSA assesses as having the greatest crash risk; * FMCSA has a broad range of goals and performance measures that it uses to provide direction to--and track the performance of--its enforcement programs, including measures of the impact of its enforcement programs on the level of carrier compliance with safety regulations and on the frequency of crashes, injuries, and fatalities; and: * FMCSA has made several refinements to its enforcement programs, such as placing more emphasis on drivers during roadside inspections based on preliminary results from a study indicating that driver errors contribute to crashes much more frequently than vehicle defects. FMCSA is also working to obtain additional information on crash risk factors, and on the costs and effectiveness of its enforcement programs and alternative approaches. This information will help FMCSA further refine and set priorities for its programs, thereby addressing deficiencies that we identified in 1999 and 2000. Although FMCSA has a broad range of goals and performance measures for its enforcement programs, it does not measure the effect that one of its key enforcement actions--civil penalties against motor carriers--has on carriers' compliance with safety regulations. Civil penalties comprised 81 percent of FMCSA's enforcement actions against motor carriers following compliance reviews--which FMCSA considers to be its key enforcement program--during fiscal years 2002 through 2004. Without a measure of the effectiveness of civil penalties, FMCSA does not know whether or how much they are increasing carriers' compliance; therefore it lacks the information needed to make sound decisions about any changes to its use of civil penalties. While MCSAP is designed to ensure that its grants to states contribute to the agency's mission of saving lives and reducing injuries by preventing truck crashes, FMCSA's oversight of state grantees is lacking. MCSAP employs a performance-based approach that requires states to analyze their commercial vehicle data; target their grant activities to reduce crashes, deaths, and injuries; and use performance information to demonstrate how safety improvement goals are being met. However, in reviewing the safety goals of the seven states that received the largest MCSAP grants, we could not determine whether states substantially met 37 of their 61 goals (61 percent) to improve truck safety. We could not make this determination because (1) FMCSA's grant planning meetings--in which it communicates priorities and how to develop plans with quantifiable goals--were conducted for about one- fourth of the state grantees in fiscal year 2004, and even fewer state grantees had their safety plans reviewed by service centers that year; (2) many of the safety goals were missing key elements, such as quantifiable performance measures and targets, and some safety plans were missing evaluations of whether goals were substantially met; and (3) FMCSA division offices that work with states did not sufficiently monitor and ensure states' progress towards safety goals. In addition, although FMCSA requires its field division offices, its four regional service centers that support division offices, and headquarters to periodically review grant program activities for adequacy of oversight (among other issues), these reviews are only being partially completed. In this regard, FMCSA division offices reviewed 19 of the 56 state grantees (34 percent) in the past 3 years. For those division offices that did not conduct reviews, reasons included restructuring occurring within the lead agency in the state that is responsible for the grant[Footnote 4] and uncertainty about whether conducting smaller "process" reviews fulfilled the state review requirement. Furthermore, service centers conducted 15 division office reviews (29 percent) and headquarters did not review any service centers in the past 3 years. FMCSA did not conduct these reviews for a variety of reasons, including a weakened oversight role for service centers, and an almost two-thirds reduction in headquarters staffing for MCSAP activities. We are making several recommendations to improve FMCSA's ability to determine and demonstrate the effectiveness of its enforcement approach, and to make adjustments to this approach when needed. For example, we are recommending that FMCSA measure the effectiveness of its civil penalties against motor carriers. We are also making recommendations to improve FMCSA's oversight of MCSAP grants to help ensure they lead to safety improvements. For example, we are recommending that FMCSA ensure that existing planning and oversight mechanisms are carried out. We provided a draft of this report to the Department of Transportation for review and comment and received comments from FMCSA officials. FMCSA generally agreed with the report's findings and agreed to consider our recommendations. FMCSA offered several corrections, which we incorporated in this report. Background: The trucking industry is an important component of the nation's economy. Of all manufactured goods and raw materials shipped across the country, close to three-fourths of their value and nearly two-thirds of their tonnage are transported by trucks. In 2002, trucks transported more than $6.2 trillion and 7.8 billion tons of goods and materials (these are the latest data available). The increased demand for transporting freight and the deregulated nature of the trucking industry have resulted in a growing industry. In recent years, the numbers of carriers, trucks, drivers, and vehicle miles traveled have been increasing. (See fig. 1.) The motor carrier industry comprises approximately 677,000 interstate carriers operating some 7.9 million large trucks and employing several million drivers. Carriers frequently enter and exit the industry; in 2004, the industry had a net gain of approximately 31,000 interstate carriers. Figure 1: Growth in the Number of Interstate Motor Carriers, 1995 through 2004: [See PDF for image] Note: The numbers of carriers are estimates. [End of figure] There are more fatalities each year resulting from passenger vehicle crashes than from truck crashes; however, the likelihood that a fatality will occur is greater for crashes that involve large trucks. In 2004, 5,190 fatalities resulted from large truck crashes while 38,531 fatalities resulted from passenger vehicle crashes (based on preliminary data). Although large trucks are involved in 4 percent of all accidents, they contribute to 12 percent of the fatalities. As a result, fatality rates--the number of fatalities per 100 million vehicle miles traveled--involving truck crashes are consistently higher when compared to passenger-vehicle-only crashes. (See fig. 2.) Figure 2: Fatality Rates Involving Large Truck and Passenger Vehicle Crashes, 1995 through 2004: [See PDF for image] Note: Fatality rates for 2004 are based on preliminary data. [End of figure] Large-truck fatality rates have generally been falling since the mid- 1990s--from 2.76 in 1995 to 2.29 in 2004 (based on preliminary data). Although there has been a reduction in the fatality rate, the number of fatalities has increased in each of the last two years, reaching 5,190 in 2004. According to FMCSA, truck crashes result in costs totaling more than $20 billion annually. In an attempt to reduce the number and severity of crashes involving large trucks, FMCSA was created by the Motor Carrier Safety Improvement Act of 1999. FMCSA assumed almost all of the responsibilities and personnel of the Federal Highway Administration's Office of Motor Carriers. The agency's primary mission is to reduce crashes, injuries, and fatalities involving large trucks by (1) issuing, administering, and enforcing federal motor carrier safety regulations and hazardous materials regulations; (2) providing education and outreach for motor carriers and drivers regarding the federal motor carrier safety regulations and hazardous materials regulations; (3) gathering and analyzing data on motor carriers, drivers, and vehicles; (4) developing information systems to improve the transfer of data; and (5) researching new methods and technologies to enhance motor carrier safety. FMCSA's goal is to reduce the fatality rate for crashes involving trucks by 41 percent between 1996 and 2008, from 2.81 to 1.65 fatalities per 100 million vehicle miles traveled. FMCSA was on track to meet this goal through 2002, when the fatality rate of 2.30 was slightly below the agency's interim target of 2.32. However, the 2003 fatality rate of 2.31 was higher than the agency's interim target of 2.19, and in 2004 the gap between the fatality rate (based on preliminary data)--2.29--and the agency's interim target--2.07--grew. FMCSA has an additional goal to reduce the number of serious, reportable hazardous materials incidents involving trucks by 20 percent (from 463 to 370) between 2000 and 2010.[Footnote 5] FMCSA programs are intended to contribute to these goals by addressing safety in motor carrier operations through identifying and enforcing safety regulations that target high-risk carriers and large-commercial- truck drivers, improving safety information systems and commercial motor vehicle technologies, strengthening commercial motor vehicle equipment and operating standards, and increasing safety awareness. Although each of these activities plays a role in FMCSA's overall safety approach, FMCSA considers enforcement to be its primary approach for reducing the number of crashes, fatalities, and injuries involving trucks. Most of FMCSA's enforcement programs focus on two parties that greatly influence the safety of truck operations--motor carriers and truck drivers. (See table 1.) In each fiscal year from 1996 through 2004, FMCSA, on average, initiated 3,800 enforcement cases against motor carriers following compliance reviews, and FMCSA and its MCSAP partners placed, on average, about 180,000 drivers and about 450,000 vehicles out of service following roadside inspections. In addition to its traditional enforcement approach, FMCSA, in conjunction with the National Highway Traffic Safety Administration, recently funded a pilot program in Washington state that combined education and enforcement with the purpose of improving the driving behavior of passenger-vehicle drivers when in the vicinity of trucks. Table 1: FMCSA's Enforcement Programs: Commercial carriers: Program: Compliance reviews; Description: On-site reviews of carriers' records and operations to determine compliance with safety regulations that address areas such as alcohol and drug testing of drivers, insurance, crashes, driver qualifications, drivers' hours of service, vehicle maintenance and inspections, and transportation of hazardous materials. Program: New entrant safety audits; Description: Audits conducted on new interstate carriers within their first 18 months of registration to ensure that they are knowledgeable about the safety regulations prior to receiving permanent registration. Although the emphasis of new entrant safety audits is on education, FMCSA can apply enforcement actions when new entrants are found to not be in compliance with safety regulations. Program: Border safety audits; Description: Audits conducted on all Mexican-domiciled carriers within their first 18 months of registration to certify that they are following safety practice and performance guidelines prior to receiving permanent certificates of registration. Commercial vehicles and drivers: Program: Roadside inspections; Description: Inspections of drivers or vehicles conducted at the roadside to determine compliance with safety regulations that address such areas as driver's license, alcohol and drug use, hours of service, brakes, turn signals, head lights, and tires. Program: Traffic enforcement; Description: Enforcement against truck drivers who violate traffic safety laws. Program: Passenger vehicle drivers. Program: Share the Road Safely[A]; Description: A pilot program in Washington state that combines education and enforcement activities to increase the safety of passenger vehicle drivers when driving in proximity to commercial vehicles. Source: GAO analysis of FMCSA data. [A] Prior to the pilot program, Share the Road Safely relied solely on education to increase the safety of passenger vehicle drivers. [End of table] In addition, FMCSA has several information systems and a program to help it identify high-risk carriers and drivers and to assist it in enforcing safety regulations.[Footnote 6] * The Safety Status Measurement System evaluates the safety of carriers by analyzing four broad categories--accidents, drivers, vehicles, and safety management--and assigns an overall score to the carrier. FMCSA then targets its compliance reviews at carriers that pose the greatest risk. * The Performance and Registration Information Systems Management program (PRISM), a grant-funded program, is a federal and state cooperative effort that, by revoking or denying registration of carriers' vehicles, aims to ensure that carriers placed out of service by FMCSA do not operate. * The Enforcement Management Information System is a database used by FMCSA to monitor, track, and store information related to enforcement actions. * The Motor Carrier Management Information System is an information system used by FMCSA as a central repository to compile inspection, crash, compliance review, safety audit, and registration data. * Gotham is a web-based system that compiles information from the Motor Carrier Management Information System, the Enforcement Management Information System, and field offices to supply information and performance measures to field managers. When FMCSA discovers a violation of the safety regulations, it may use one of several enforcement actions depending on the nature of the violation. (See table 2.) An enforcement action may require the violating party to correct the unsafe practice or operation, pay a civil penalty, or suspend operations. Table 2: FMCSA's Enforcement Actions: Enforcement action: Compliance order; Description: Directs a carrier to comply with the safety regulations. Enforcement action: Civil penalty; Description: Imposes a monetary penalty on a carrier that violates a safety regulation. Enforcement action: Out-of-service order; Description: Orders a driver or vehicle out of service for posing an imminent hazard to safety, or orders a carrier to cease all or part of its operation for having imminently hazardous conditions or operations, or for being unfit. Enforcement action: Order-to-cease operations; Description: Prohibits a carrier from operating in interstate commerce for failing to pay a civil penalty assessed by FMCSA. Source: GAO analysis of FMCSA data. [End of table] FMCSA has approximately 1,050 full-time equivalent employees, of which nearly 850 work at 52 division offices throughout the U.S. and its territories. The division offices oversee 56 MCSAP grantees--one grantee in each of the fifty states, one in the District of Columbia, and one each in the U.S. territories of American Samoa, Guam, the Northern Marianas, Puerto Rico, and the Virgin Islands.[Footnote 7] Much of the work carried out in the field involves conducting reviews either at the roadside or at a carrier's place of business. For fiscal year 2006, FMCSA was appropriated $501 million. Over half of these funds are slated for distribution to states in the form of grants, the largest under MCSAP. MCSAP provides financial assistance to states to reduce commercial motor vehicle-involved accidents, fatalities, and injuries through consistent, uniform, and effective commercial motor vehicle safety activities. Initially, MCSAP primarily funded state roadside inspections as a method of improving commercial motor vehicle safety. However, the program has evolved over the past two decades to fund several other safety initiatives in support of its goal, including compliance reviews, traffic enforcement, new entrant safety audits, border grants, and other safety initiatives.[Footnote 8] The recently- enacted Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users authorized an average annual funding level of $200 million for MCSAP--more than twice the amount available in the previous authorization. The responsibility for the administration and the oversight of MCSAP is shared among the various levels of FMCSA. Division offices in each state have the primary responsibility for overseeing state programs. They work closely with the states to develop commercial vehicle safety plans and monitor the states' activities to ensure program goals are being met, and are also responsible for ensuring that grant funds are spent for reimbursable expenses. MCSAP responsibilities of the regional service centers and headquarters differ from those of the division offices. The service centers act as an intermediary between the division office and headquarters. They assist in clarifying policy for the division offices and they organize training and goal-setting meetings related to the grant program. Headquarters responsibilities center on establishing and communicating agency priorities for MCSAP, issuing and updating MCSAP policy guidance, and carrying out financial management activities. FMCSA's Enforcement Programs and Actions Have Been Stronger Since 2000: In response to criticisms in 1999 that its enforcement programs were ineffective at improving safety, FMCSA has strengthened its enforcement approach. Its enforcement approach has evolved to supplement strong enforcement with additional measures to encourage carriers to comply with safety regulations. FMCSA Committed to a Stronger Enforcement Approach: During the 1990s, FMCSA believed that adopting a partnering approach with the trucking industry--marked by an increased emphasis on education and less emphasis on traditional enforcement programs--would lead to improved large truck safety. However, while implementing this approach, the fatality rate from crashes involving large trucks continued to increase. The Department of Transportation's Inspector General criticized FMCSA in a report stating that the approach was ineffective and that the agency needed to emphasize traditional enforcement programs.[Footnote 9] Moreover, we reported then that FMCSA lacked an understanding of the effectiveness of its enforcement programs.[Footnote 10] In response to the criticism, FMCSA publicly committed to strengthen its enforcement approach by committing to increase its emphasis on enforcement programs and actions. Through the Transportation Equity Act for the 21st Century in 1998 and the Motor Carrier Safety Improvement Act of 1999, Congress broadened the government's enforcement authority, allowing it to implement this new emphasis. This legislation made it easier for FMCSA to revoke carriers' operating authority. Furthermore, the legislation increased the allowable civil penalty from $1,000 to $10,000 for certain violations. Subsequently, the agency has come to believe that combining strict enforcement with education and outreach will lead to greater improvements in safety. This approach seeks to identify higher risk operators and apply education and enforcement where needed. For example, the agency has a program--geared toward new entrants to the industry--to encourage awareness of the federal motor carrier safety regulations. However, FMCSA officials told us that the agency is committed to applying strict enforcement to egregious offenders who demonstrate non-compliance with the regulations or unsafe driving practices. FMCSA has also made progress in assessing the effects of its enforcement programs on safety in terms of reducing crashes, injuries, and fatalities. We discuss these efforts in more detail below. Finally, FMCSA has restructured its organization to allow enforcement and education programs to work more closely together. For example, FMCSA moved the program manager for education and outreach from the Office of Communications to the Office of Enforcement and Program Delivery in order to increase collaboration between FMCSA's education and enforcement staff. FMCSA officials have emphasized that although they are taking steps to increase the role of education, they do not plan to return to the partnering approach--i.e., in which education is the primary focus--that characterized the mid-1990s. FMCSA's Programs Have Mirrored Its Stronger Enforcement Approach: The number of compliance reviews was higher in 2004 than in 1998. Before the creation of FMCSA, the number of compliance reviews conducted dropped 31 percent between fiscal years 1995 and 1998--from approximately 9,200 to 6,400. (See fig. 3.) Demonstrating its commitment in 1999 to follow a stronger enforcement approach, FMCSA set and met its goal of doubling the number of compliance reviews. In this respect, the number of compliance reviews more than doubled from approximately 6,400 in 1998 to more than 13,400 in 2001.[Footnote 11] The number of compliance reviews began to decrease in fiscal year 2002, largely as the result of additional homeland security responsibilities to review hazardous materials carriers in the wake of the September 11, 2001, terrorist attacks on the United States. The agency completed almost 31,000 homeland security-related visits in fiscal year 2002. The number of these visits decreased to less than 2,000 in fiscal year 2003, when FMCSA began targeting the reviews at the most vulnerable carriers as identified by the agency's risk assessment. Figure 3: Number of Compliance Reviews Conducted and Percentage of the Motor Carrier Industry Receiving Compliance Reviews, 1995 through 2004: [See PDF for image] [End of figure] Another factor affecting FMCSA's ability to conduct compliance reviews was its responsibility for implementing the new entrant audit program in 2003. From fiscal year 2003 through August 2005, FMCSA conducted almost 54,500 new entrant audits. Despite these additional responsibilities of conducting visits of hazardous materials carriers and new entrants, the agency has still conducted more compliance reviews than were carried out in the late 1990s. In 2004, the agency conducted approximately 11,300 compliance reviews, 77 percent more than in fiscal year 1998. Despite an increase in the number of compliance reviews, the agency has still been unable to review a larger proportion of the industry's carriers. This is because the industry has grown faster than has FMCSA's ability to conduct compliance reviews. (See figs. 1 and 3.) * In 1995, FMCSA performed about 9,200 compliance reviews, reaching about 2.7 percent of the nation's estimated 350,000 carriers.[Footnote 12] * In 1998, when FMCSA conducted its lowest number of compliance reviews--about 6,400--the industry had grown almost 40 percent to an estimated 480,000 carriers. As a result, FMCSA's compliance reviews covered only about 1.3 percent of the industry. * Then, in 2000, as FMCSA doubled the number of compliance reviews, the number of carriers increased by almost 80,000 from the 1998 level, resulting in 2.4 percent of carriers receiving compliance reviews in 2000. * In fiscal year 2004, FMCSA reviewed about 1.7 percent of the carriers--a smaller proportion than the proportion reviewed in the mid- 1990s--despite increasing the number of compliance reviews by 77 percent from the late 1990s. This is because the trucking industry had continued to grow steadily to about 677,000 carriers in 2004. In fiscal year 2005, FMCSA expressed its intent to once again increase the number of compliance reviews. As discussed in more detail later in this report, FMCSA is also exploring potential changes to its compliance review program, in part because it is concerned that the program reaches only about 2 percent of carriers each year. Roadside inspections have also increased. In the late 1990s, FMCSA and the states continued to increase the number of roadside inspections, even as compliance reviews were decreasing. During that time period, FMCSA and states nearly doubled the number of roadside inspections conducted from 1.3 million to 2.3 million.[Footnote 13] (See fig. 4.) The number of roadside inspections increased steadily through 2000, when FMCSA and state inspectors conducted about 2.5 million inspections. In its fiscal year 2005 and 2006 budgets, FMCSA set its goal for roadside inspections at 2 million--about 1 million less than were conducted in 2004--bringing the number of expected roadside inspections back to pre-2000 levels. According to an FMCSA official, FMCSA set these lower goals in 2005 and 2006 with the anticipation that the number of roadside inspections would decrease as states shifted resources from roadside inspections to conduct increasing numbers of audits of new entrant carriers. However, the anticipated decrease in roadside inspections did not occur because some states hired additional staff to conduct the audits of new entrants and therefore did not have to use their roadside inspection staff to conduct these audits. An FMCSA official also told us that the agency is no longer seeking to increase the annual number of roadside inspections based on its analysis indicating that the inspection program is no longer leading to annual increases in the industry-wide level of compliance with safety regulations. Figure 4: Roadside Inspections Conducted and Planned, 1996 through 2006: [See PDF for image] Note: 1995 data are not included because FMCSA determined that they are not reliable. FMCSA does not control the total number of inspections conducted; states may choose to do more inspections. [End of figure] FMCSA intends to make new entrant audits more enforcement-oriented. The Motor Carrier Safety Improvement Act established the new entrant audit program to educate carriers that are new to the industry about safety regulations and encourage their compliance. This act was in response to a 1988 FMCSA-commissioned study that pointed to a higher rate of violations of safety regulations and higher crash rates among carriers that had recently entered the motor carrier industry. In fiscal year 2003, the agency began implementing the new entrant audit program and conducted 7,000 audits. As FMCSA fully implemented the program in 2004, this number increased to more than 25,000. FMCSA believes that in the audit's current form, the "pass rate" does not accurately reflect new entrant carriers' level of safety. Currently, carriers can fail one-third of the audit sections and still pass the audit, resulting in a pass rate of more than 99 percent.[Footnote 14] FMCSA plans to improve the program by employing stricter criteria for carriers and increasing the threshold for carriers to pass, demonstrating the agency's approach of using strict enforcement where needed. The agency plans to publish a proposed regulation for public comment in March 2006. FMCSA's Enforcement Actions Have Increased: FMCSA has initiated more enforcement cases and identified higher numbers of serious violations. FMCSA calculates a "rate of enforcement," which is the percent of compliance reviews that result in an enforcement case. Since 2000, the rate of enforcement has increased from 30 percent to 46 percent of compliance reviews. (See fig. 5.) Further, the number of serious violations (FMCSA calls these "acute violations") that FMCSA has identified is consistently higher than in the 1990s. In 1998, the agency identified approximately 710 serious violations of the regulations. This number increased by 64 percent to approximately 1,160 violations in 2000. The number of serious violations peaked in 2003 when FMCSA identified more than 1,800 violations. According to an FMCSA official, the increase in the number of serious violations identified resulted from FMCSA's improved targeting of compliance reviews of high-risk carriers. In 2004, the number dropped to about 1,500--out of about 14,800 total violations--but was still more than double the 1998 number. Figure 5: Number of Serious Violations Found and Rate of Enforcement, 1995 through 2004: [See PDF for image] [End of figure] FMCSA has applied its authority to place carriers out of service. Between 2001 and 2004, the number of carriers prohibited from operating increased from about 170 to about 1,700. (See fig. 6.) Before 2001, FMCSA prohibited motor carriers from operating if the agency found that the carrier posed an imminent hazard. FMCSA has explained that because the definition of "imminent hazard status" was vague and limited in scope, the agency ordered very few motor carriers to cease operating. In 2001, based on authorities given it by the Motor Carrier Safety Improvement Act, FMCSA began prohibiting carriers from operating if they failed to correct operational problems or pay a civil penalty. Figure 6: Number of Carriers Prohibited from Operating in Interstate Commerce by FMCSA, 1995 through 2004: [See PDF for image] Note: This includes carriers that are closed as a result of both violations to the regulations and failure to pay assessed civil penalties. [End of figure] The size of proposed civil penalties was higher in 2004 than in 1998.[Footnote 15] From 1995 through 1998, the average civil penalty proposed for each violation decreased by 10 percent from $910 in 1995 to $820 in 1998.[Footnote 16] (See fig. 7. All amounts are in 2004 dollars.) A key criticism by the Department of Transportation's Inspector General during that time was that carriers had begun to see these civil penalties as little more than a cost of doing business. In response, FMCSA pledged to levy larger civil penalties and, in 1999, implemented the authorities that Congress gave the agency. From 1998 to 2000, the average civil penalty levied per violation increased by more than 80 percent to almost $1,500. In recent years, FMCSA has modified its view of civil penalties, citing that they are only one tool to reach compliance. Accordingly, the average civil penalty for each violation has dropped slightly to approximately $1,400 in 2004, but this represents a drop of about 3 percent and does not bring the amount of civil penalties back to the levels of the late 1990s. While the civil penalties have been higher since 2000, the agency still has considerable latitude under law to assess larger civil penalties.[Footnote 17] As discussed later in this report, FMCSA does not know whether or how much its civil penalties are increasing carriers' compliance, and therefore the agency has a limited ability to determine whether the dollar amounts of the penalties are at the most effective levels. Figure 7: Average Civil Penalty Assessed per Violation, by Type of Violation, Fiscal Years 1995 through 2004 (In 2004 Dollars): [See PDF for image] Note: Numbers may differ from those reported by FMCSA because they have been adjusted for inflation. The category of non-recordkeeping violations did not exist before 1998. [End of figure] FMCSA has assessed higher civil penalties for more serious offenses. The agency has four categories of violations: acute, critical, non- record keeping, and record keeping.[Footnote 18] * Acute violations are so severe that FMCSA will require immediate corrective actions by a motor carrier regardless of the overall safety status of the motor carrier. An example of an acute violation is a carrier failing to implement an alcohol or drug testing program for drivers. * Critical violations are serious, but less severe than acute violations and most often point to gaps in carrier management or operational controls, such as not maintaining records of driver medical certificates. * Non-record keeping violations are related directly to driver or carrier actions. An example is a driver operating a motor carrier under the influence of drugs or alcohol. * Record keeping violations involve a driver or carrier failing to maintain required documentation. An example of a record keeping violation is failure of the carrier to maintain a record of all accidents for one year. In line with FMCSA's commitment to stricter enforcement for more serious violations, since 2000, acute violations, which FMCSA identifies as the most serious violation, and non-record keeping violations have been assessed penalties that are about five times higher than record keeping violations, as shown in figure 7. Further, the average civil penalty per violation of critical violations has consistently been more than twice the average civil penalty per record keeping violation. We also assessed the degree to which FMCSA reduced its initial civil penalty assessment.[Footnote 19] We found that the average final assessment has been 80 percent or more of the initial assessment during this past decade.[Footnote 20] (See fig. 8.) Further, since 2000, the first fiscal year after FMCSA promised to be stronger on enforcement, acute violations were reduced slightly less than overall violations. Figure 8: Final Assessment as Compared to Initial Assessment, Fiscal Years 1995 through 2004: [See PDF for image] [End of figure] In recent years, FMCSA has applied a new approach to civil penalties. With first-time offenders and other carriers with less serious violations, FMCSA has reduced civil penalties in cases where the carrier agrees to come into compliance and make additional safety- related improvements. The agency has specified that any carrier receiving a reduction in its civil penalty must remain in compliance for several years. If the carrier violates safety rules again, it is then responsible for paying the entire originally assessed civil penalty or FMCSA will place the carrier out of service. FMCSA does not currently maintain information on the subpopulation of first time offenders, and therefore it has not assessed the results of this new policy. In November 2005, FMCSA began issuing and tracking notices to first-time offenders and other carriers with less serious violations that describe the violations and the corrective actions required, without assessing a civil penalty. This tracking should give FMCSA the ability to assess the results of the policy. FMCSA Has a Well-Defined Enforcement Approach and Has Efforts Underway to Help It Refine and Set Priorities for Its Enforcement Programs: To a large extent, FMCSA follows key effective management practices in making decisions about its enforcement approach. FMCSA's enforcement approach addresses major risk factors that contribute to or cause crashes, and FMCSA targets its enforcement resources at the motor carriers that it assesses as having the greatest crash risk. FMCSA has a broad range of goals and performance measures that it uses to provide direction to--and track the performance of--its enforcement programs, but it does not have a measure of the effectiveness of one of its primary enforcement actions: civil penalties against motor carriers. FMCSA has made several refinements to its enforcement programs based on agency studies of factors that affect crash risk and on evaluations of the effectiveness of the programs themselves. FMCSA is also working to obtain additional information on crash risk factors, and on the costs and effectiveness of its enforcement programs, needed to help it further refine and set priorities for the programs. FMCSA's Enforcement Approach Addresses Many of the Major Factors that Cause or Contribute to Truck Crashes and Targets High-Risk Carriers: FMCSA's enforcement approach includes goals and programs that address many of the major factors that cause or contribute to truck crashes. (See table 3.) For example, motor carrier operations--including hiring unsafe drivers and inadequate vehicle maintenance practices--have been identified as one of a number of interacting factors that can contribute to crashes. To address this, FMCSA conducts compliance reviews of motor carriers' operations and takes enforcement actions against carriers found to have violations of the safety regulations. FMCSA also addresses vehicle condition and driver factors (such as fatigue) by conducting roadside inspections of drivers and their trucks. Another factor that has been identified is unsafe driving by passenger-vehicle drivers in the vicinity of trucks. To address this factor, FMCSA and the National Highway Traffic Safety Administration are jointly funding a pilot program in Washington state in 2005 to educate passenger-vehicle drivers about how to drive safely in the vicinity of trucks and to take enforcement action against drivers who drive unsafely.[Footnote 21] Table 3: How FMCSA's Enforcement Approach Addresses Many of the Major Factors that Cause or Contribute to Truck Crashes: Major factor that causes or contributes to truck crashes: Motor carrier operations; FMCSA's enforcement approach: Goal: Improve the safety performance of high-risk carriers; FMCSA's enforcement approach: Program: Compliance reviews; PRISM. FMCSA's enforcement approach: Goal: Improve the safety performance of new entrant carriers; FMCSA's enforcement approach: Program: Safety audits of new entrant carriers[A]. FMCSA's enforcement approach: Goal: Truck driver performance: Ensure that Mexican carriers operating in the U.S. comply with safety regulations; FMCSA's enforcement approach: Program: Truck driver performance: Safety audits of Mexican carriers. Major factor that causes or contributes to truck crashes: Truck driver performance; FMCSA's enforcement approach: Goal: Ensure that all commercial motor vehicle drivers are fully qualified, safe, alert, and healthy; FMCSA's enforcement approach: Program: Roadside inspections; traffic enforcement. Major factor that causes or contributes to truck crashes: Vehicle performance; FMCSA's enforcement approach: Goal: Ensure that commercial motor vehicles have optimum safety performance; FMCSA's enforcement approach: Program: Roadside inspections. Major factor that causes or contributes to truck crashes: Passenger- vehicle driver performance; FMCSA's enforcement approach: Goal: Improve the safety and performance of non-commercial drivers with respect to trucks; FMCSA's enforcement approach: Program: Share the Road Safely. Source: GAO analysis of FMCSA information. [A] This program currently emphasizes education of motor carriers over enforcement, but FMCSA intends to increase the enforcement emphasis of the program. [End of table] Because FMCSA's resources do not allow it to conduct compliance reviews of each of the estimated 677,000 motor carriers and roadside inspections of each of the industry's millions of vehicles and drivers, FMCSA's enforcement approach targets high-risk carriers and their vehicles and drivers. As discussed above, FMCSA uses information about carriers' safety performance (including crash history and results of roadside inspections and compliance reviews) to identify unsafe carriers that it then targets for additional compliance reviews; many states also use the information to target their roadside inspections of drivers and their trucks.[Footnote 22] Because carriers that have drivers with more convictions tend to have higher crash rates, FMCSA is working to incorporate these driver conviction histories into its targeting system. FMCSA Has a Broad Range of Enforcement Goals and Performance Measures but It Does Not Measure the Effectiveness of Its Civil Penalties: FMCSA has a broad range of goals and related performance measures that it uses to provide direction to--and track the progress of--its enforcement programs, but it does not measure the effect that one of its key enforcement actions--civil penalties against motor carriers-- has on carriers' compliance with safety regulations. Without such a performance measure, FMCSA does not know whether or how much its civil penalties are increasing carriers' compliance, and it lacks the information needed to make sound decisions about any changes to its use of civil penalties. FMCSA's budget request for fiscal year 2006 includes a "logic model" that describes how its enforcement programs are expected to result in the desired end outcome of reductions in truck crashes, injuries, and fatalities.[Footnote 23] (See fig. 9.) The model follows one type of structure for such models, in which outputs (in this case, enforcement activities and actions) result in intermediate outcomes (increased compliance with safety regulations), which in turn result in end outcomes (increased safety). Figure 9: Logic Model of How FMCSA's Enforcement Approach Contributes to Truck Safety: [See PDF for image] [End of figure] For each component of its enforcement program logic model--outputs, intermediate outcomes, and end outcomes--FMCSA has various goals and performance measures. (See table 4.) The Government Performance and Results Act of 1993 calls for agencies to establish goals, including outcome goals, and performance measures for both outputs and outcomes, and we have reported that it is a useful practice for federal programs to set intermediate outcome goals in order to help show a program's contribution to desired end outcomes.[Footnote 24] FMCSA describes some of its enforcement goals and performance measures in its annual budget requests that it submits to Congress, and it describes some of its other enforcement goals and performance measures in its 2003 report entitled Measuring the FMCSA's Safety Objectives from Year 2000 to 2002. FMCSA intends to update this report annually beginning in February 2006. In addition, FMCSA reports its progress on several of its enforcement performance measures in a quarterly progress report that it posts on its Web site. Finally, FMCSA uses an information system to internally track several performance measures related to its management of enforcement cases. Table 4: Examples of FMCSA's Enforcement Program Goals and Performance Measures: Output: Goal: Complete 10,000 federally conducted compliance reviews of motor carriers in fiscal year 2006; Goal is numerically specified: Yes; Performance measure(s): Number of federally conducted compliance reviews of motor carriers completed. Goal: Reduce the agency's backlog of enforcement cases; Goal is numerically specified: No; Performance measure(s): Number of enforcement cases that have remained open for 6 months or more. Intermediate outcome: Goal: Increase the safety performance of the worst offending motor carriers to meet the norm; Goal is numerically specified: No[A]; Performance measure(s): FMCSA's performance measures for this goal include: * For poor-performing motor carriers collectively, a numerical rating based on the frequency and severity of violations of driver-related acute or critical regulations cited at compliance reviews; * Percentage of carriers previously rated "conditional" that improve their rating to "satisfactory" on a follow-up compliance review. Goal: Ensure that all commercial motor vehicle drivers are fully qualified, safe, alert, and healthy; Goal is numerically specified: No; Performance measure(s): FMCSA's performance measures for this goal include: * Percentage of inspections that resulted in a driver being placed out of service; * Number of driver out-of-service violations per inspection, with more recent violations receiving more weight and a penalty applied for instances in which drivers were found to be violating out-of-service orders. End outcome: Goal: Reduce crashes, injuries, and fatalities by conducting compliance reviews; Goal is numerically specified: No; Performance measure(s): Number of crashes, injuries, and fatalities avoided by conducting compliance reviews. Source: GAO analysis of FMCSA information. [A] Although the goal suggests that FMCSA could use a numerical measure of the safety performance of well-performing motor carriers as a target level of performance for the worst offending motor carriers, the agency does not do so. [End of table] FMCSA has output goals for its enforcement programs that specify how many of certain types of activities, such as compliance reviews and roadside inspections, it intends to complete. FMCSA also tracks many performance measures related to its use of enforcement actions, such as the number of vehicles, drivers, and motor carriers it places out of service; the number and average dollar amount of civil penalties that it assesses against motor carriers; and the average dollar amount of civil penalties that have been reduced during FMCSA's negotiations with carriers. FMCSA does not set goals linked to these performance measures because it believes that such goals could bias the decisions of enforcement personnel. We agree that such goals could result in undesirable behavior. FMCSA has set an output goal related to another aspect of its enforcement actions--how long it takes the agency to close its enforcement cases.[Footnote 25] In May 1999, FMCSA set a goal of eliminating by January 1, 2000, the backlog of 1,200 enforcement cases that had been forwarded by one of FMCSA's regional offices to FMCSA's headquarters office.[Footnote 26] As the Department of Transportation's Inspector General has noted, a large backlog undermines the integrity of the enforcement process, since (1) FMCSA considers only closed enforcement cases when targeting motor carriers for a compliance review, and therefore high-risk motor carriers are less likely to be selected if they have an open enforcement case, and (2) FMCSA assesses smaller civil penalties against carriers with open cases than those with closed cases, and therefore it may not assess appropriate civil penalty amounts against carriers with multiple enforcement cases. FMCSA did not meet its goal of eliminating the backlog of cases in headquarters, but it did reduce the backlog by a commendable 70 percent--to 363 cases in September 1999. As of October 2005, the backlog in headquarters stood at 140 cases, and the service centers that replaced FMCSA's regional offices had an additional combined backlog of 667 cases. Reducing these backlogs is one of FMCSA's four highest priorities for improving commercial motor vehicle safety in fiscal years 2005 and 2006. However, FMCSA does not have a goal that specifies how much it would like to reduce the backlogs or by what date, as required by the Motor Carrier Safety Improvement Act. FMCSA has several intermediate outcome goals and performance measures related to the effects that its enforcement programs have on increasing carriers' and drivers' compliance with safety regulations. However, FMCSA does not measure the effect that one of its key enforcement actions--civil penalties against motor carriers--has on carriers' compliance or safety.[Footnote 27] In addition, FMCSA's intermediate goals do not include numerical targets that would help the agency plan its programs and make adjustments should it not meet the targets. FMCSA's intermediate outcome goals include: (1) ensuring that commercial motor vehicles have optimum safety performance; (2) ensuring that all commercial motor vehicle drivers are fully qualified, safe, alert, and healthy; and (3) increasing the safety performance of the worst offending motor carriers to meet the norm. FMCSA has many performance measures related to these goals, including ones that reflect the average numbers of vehicle-out-of-service violations and driver-out-of-service violations per roadside inspection and the percentage of compliance reviews with no violations of acute or critical regulations. The performance measures that address motor carrier performance capture the effects of FMCSA's compliance reviews, but they do not indicate the effect on compliance of civil penalties against carriers that result from compliance reviews. This is because the performance measures do not distinguish between those compliance reviews that result in a civil penalty and those that do not, and either type of compliance review can increase carriers' compliance.[Footnote 28] Without a measure of the effectiveness of its civil penalties, FMCSA does not know whether or how much the civil penalties are increasing carriers' compliance, and it lacks the information needed to make sound decisions about any changes to its use of civil penalties. Several years ago, FMCSA did attempt to develop measures of the effectiveness of its enforcement actions, including civil penalties, but it was not satisfied with the results of the effort. FMCSA's performance measures of the end outcomes of its enforcement programs are crashes, injuries, and fatalities avoided due to compliance reviews, roadside inspections, and traffic enforcement--the difference between the actual numbers of crashes, injuries, and fatalities, and how many would have occurred in the absence of these enforcement programs. Because the numbers that would have occurred in their absence cannot be observed, FMCSA uses analytical models to estimate the programs' impacts.[Footnote 29] Using its compliance review impact model, FMCSA estimates that about 9,200 compliance reviews conducted in 2002 prevented about 1,400 crashes, about 1,100 injuries, and about 60 fatalities in the 12 months following the compliance reviews.[Footnote 30] Using its combined roadside inspection and traffic enforcement impact model, FMCSA estimates that about 3 million roadside inspections (about 800,000 of which included enforcement of traffic laws) conducted in 2004 prevented about 19,000 crashes, about 14,000 injuries, and about 700 fatalities. According to an FMCSA official, FMCSA also plans to develop a model to estimate the impact of new entrant safety audits. FMCSA's Enforcement Goals Do Not Describe Expected Contributions to FMCSA's Fatality Rate Goal: In addition to goals for its enforcement programs, FMCSA has a primary safety goal that is agencywide--that is, FMCSA seeks to achieve the goal through its various efforts, including enforcement, education and outreach, and research and technology. FMCSA's primary safety goal is to reduce the rate of fatalities involving large trucks per 100 million miles of truck travel by 41 percent from 2.81 in 1996 to 1.65 in 2008. FMCSA expects its enforcement programs, as the agency's primary means of reducing truck crashes, to help meet the fatality rate goal. Furthermore, for three of its enforcement programs (compliance reviews, roadside inspections, and traffic enforcement), FMCSA has goals for avoiding fatalities, and, as mentioned above, it also has measures of how many fatalities these programs avoid. However, the goals do not describe the program's expected contributions to the fatality rate goal. We have reported that program goals should demonstrate programs' contributions to meeting agencywide goals.[Footnote 31] Currently, the goals state that the programs are expected to avoid fatalities, but they do not include targets that specify how many fatalities are expected to be avoided each year. However, in order to demonstrate expected contributions to reducing the fatality rate, the goals would need to have numerical targets that increase from year to year; these targets should also be commensurate with the resources the agency expects to receive. Similarly, in order to demonstrate the enforcement programs' estimated contributions to any reductions in the fatality rate, FMCSA would need to compare its estimates of fatalities avoided from year to year. For example, based on FMCSA's estimates of fatalities avoided, its enforcement programs avoided 19 more fatalities in 2002 compared to 2001 (848 fatalities avoided in 2002 compared to 829 fatalities avoided in 2001). This estimated increase in fatalities avoided represents the estimated contribution that FMCSA's enforcement programs made to the overall reduction of 172 truck-crash-related fatalities that occurred from 2001 to 2002 (from 5,111 fatalities in 2001 to 4,939 fatalities in 2002). Turning to fatalities per 100 million truck miles traveled, the rate declined from 2.45 in 2001 to 2.30 in 2002. Based on the estimated increase in fatalities avoided by FMCSA's enforcement programs in 2002 compared to 2001, these programs contributed 0.02 to the total decrease of 0.14 in the fatality rate.[Footnote 32] Because we did not assess the quality of these models, we are not suggesting that FMCSA's enforcement programs had these illustrated results. Rather, this is an example of the kind of analysis FMCSA could employ in order to demonstrate its enforcement programs' estimated contributions to any reductions in the fatality rate. FMCSA Has Made Several Refinements to Its Enforcement Programs and It May Make Additional Refinements Depending on the Outcomes of Studies: FMCSA has made several refinements to its enforcement programs based on agency studies of factors that affect crash risk, and on evaluations of the effectiveness of the programs themselves,[Footnote 33] and it is exploring additional, potentially substantial, refinements to its enforcement approach. FMCSA plans to conduct additional analyses of crash risk factors and on the costs and effectiveness of its enforcement programs to help it further refine and set priorities for its programs. FMCSA has made several refinements to its enforcement programs based on agency studies of factors that affect crash risk. To date, only one such refinement has been based on the results of FMCSA's most substantial effort to understand the causes of, and contributing factors to, large truck crashes--its Large Truck Crash Causation Study, which FMCSA began in 2001. This study was required by the Motor Carrier Safety Improvement Act with the intent of providing information that would help FMCSA and the states identify and set priorities for programs and other measures likely to lead to significant reductions in crashes involving commercial motor vehicles. Congress also required FMCSA, in designing and conducting the study, to consult with experts on crash causation, research methods, and other relevant topics. To meet this requirement, FMCSA requested the Transportation Research Board to form a committee to review the study's design and provide advice to FMCSA on study methods. The committee expressed several concerns about FMCSA's design, and FMCSA revised the design in response to some of these concerns.[Footnote 34] Based on preliminary results from the study indicating that truck and car driver factors (such as fatigue, inattention, and misjudgment) are much more likely than vehicle factors (such as defective brakes and worn tires) to be the critical reason for crashes involving a truck and a car, FMCSA has decided to shift the focus of some of its inspections from the truck to the driver,[Footnote 35] and has called on states to institute "driver inspection strike forces" in high-crash corridors. FMCSA plans to conduct more formal analyses of data from the study over the next several years, and, depending on the results, it may make additional refinements to its enforcement programs. Other examples of refinements to its enforcement programs that FMCSA has made based on studies of factors that affect crash risks include: * Safety of new entrants: Based on studies indicating that new motor carriers had higher crash rates and lower rates of compliance with safety regulations, in 1999 Congress directed FMCSA to develop a program to ensure that new entrants would operate safely. As discussed previously, in 2003 FMCSA initiated such a program that emphasizes educating new entrants about their obligations under the safety regulations, rather than taking enforcement actions when safety deficiencies are identified. FMCSA intends to strengthen the enforcement component of the new entrant program. * Share the Road Safely: Based on studies indicating that, in fatal crashes involving a car and a truck, errors by car drivers are more frequently cited in police reports than are errors by truck drivers,[Footnote 36] in 1991, Congress directed FMCSA to develop a program to educate drivers about how to drive safely in the vicinity of trucks; in 1994, FMCSA initiated the Share the Road Safely program for this purpose. Many highway safety experts believe that programs such as Share the Road Safely are more likely to produce substantial changes in drivers' behaviors if the education efforts are combined with local law enforcement programs to increase compliance with traffic laws. FMCSA agrees that enforcement should be a part of the program, and in 2005 it began to pilot test a combined education and enforcement approach in Washington state. To its credit, FMCSA has also made several refinements to its enforcement programs based on agency evaluations of the effectiveness of the programs themselves. The Government Performance and Results Act calls for agencies to evaluate the results of their programs, and we have suggested that agencies use information on the performance of their programs to identify opportunities for improvement.[Footnote 37] Examples of refinements to its enforcement programs that FMCSA has made based on its evaluations of the effectiveness of the programs include: * Roadside inspections: Based on its analysis indicating that the inspection program is no longer leading to annual increases in the industry-wide level of compliance with safety regulations, an FMCSA official also told us that the agency is no longer seeking to increase the number of roadside inspections from year to year. * Safety of new entrants: Based on (1) its analysis indicating that new entrant safety audits--which currently emphasize education--have not had a major impact on crash rates and (2) data showing that less than 1 percent of audited new entrants were failing the audit despite the fact that many new entrants were operating without comprehensive knowledge of the requirements and without being in compliance with the safety regulations, FMCSA intends to increase the level of compliance required to pass the audit and to avoid an enforcement action. FMCSA plans to publish a proposed regulation to this effect in March 2006. * Compliance reviews: Based on its 2002 evaluation indicating that inconsistencies and bottlenecks in its compliance review process were reducing its efficiency and effectiveness, FMCSA made several changes in 2003 aimed at improving compliance review policies, procedures, training, software, and supporting carrier data. Specific examples include: * FMCSA discouraged repeat visits to high-risk motor carriers that had received unsatisfactory safety ratings during a review conducted within the past 12 months because the agency believed that not enough time had elapsed to show whether safety improvements had taken effect. * FMCSA (1) discouraged safety investigators from their earlier practice of favoring violations of drug and alcohol regulations over violations of hours of service regulations when they choose which violations to document for enforcement because crash data and FMCSA's survey of its field staff suggest that hours of service compliance is more important for safety, and (2) revised its operations manual to encourage FMCSA's state division offices to document the maximum number of parts of the regulations where major safety violations are discovered, rather than penalizing motor carriers for a few violations in a particular part at the expense of other parts. FMCSA has not evaluated the impact that these and other changes have had on the efficiency and effectiveness of the compliance review process, but an FMCSA official told us that the agency believes such an evaluation would be worthwhile and that it will consider conducting one. FMCSA is either conducting, or plans to conduct, several additional evaluations of its enforcement approach, programs, and supporting programs. Depending on the results of these evaluations, FMCSA may make refinements to each of these. For example: * Comprehensive Safety Analysis 2010: Under this initiative, FMCSA is evaluating its current approach to monitoring and assessing the safety of the motor carrier industry, including its enforcement approach, with the intent of developing and implementing new approaches. FMCSA began the initiative because it believes that the centerpiece of its current oversight and enforcement approach--the compliance review program--is resource intensive; reaches only about 2 percent of a growing motor carrier population each year; does not adequately reflect the impact that people involved in motor carriers' operations, such as managers, owners, and drivers, have on safety; and is impeded by delayed, incomplete, and inaccurate data on the safety performance of carriers. As a starting point for this initiative, FMCSA developed a preliminary proposal for a new framework for its oversight and enforcement programs. In contrast to FMCSA's current enforcement approach, which considers only the results of compliance reviews in determining whether to assess civil penalties against carriers, the proposed framework also considers, among other things, indications of unsafe driving from traffic violations, crash records, and the results of roadside inspections. FMCSA intends to refine and begin implementation of the framework over the next several years. * Roadside inspections: FMCSA is evaluating the effectiveness of alternative truck inspection strategies being utilized by states, such as building temporary inspection sites, using no fixed sites at all, mobile enforcement using wireless communications technologies, and use of sophisticated information systems and different kinds of sensing technologies. FMCSA expects the results of its evaluation to be published in December 2005. As part of a separate evaluation, FMCSA plans to review alternatives to its current approach to vehicle roadside inspections, such as requiring motor carriers to inspect their vehicles more frequently than the current requirement of once a year and having third-parties conduct inspections. * PRISM: FMCSA is evaluating the effectiveness of its PRISM program, including assessing whether states that have implemented PRISM have achieved greater safety improvements than states that have not, and identifying ways to improve the efficiency and effectiveness of the program. FMCSA plans to complete the evaluation in August 2006. * Monitoring drivers' hours of service: FMCSA is evaluating the costs and benefits of requiring electronic on-board recorders for monitoring drivers' hours of service, as required by a July 2004 federal appeals court ruling that criticized the agency for not having conducted such an assessment. * Drug and alcohol testing: In the next several years, FMCSA plans to study whether the effectiveness of the agency's efforts to ensure compliance with its regulations prohibiting certain uses of drugs and alcohol by drivers is being adversely affected by (1) how motor carriers are selecting drivers for random testing, (2) whether carriers are actually testing each driver selected, and (3) tests in which bogus samples are submitted. FMCSA may make policy, regulatory, and enforcement changes depending on the findings of the study. * Share the Road Safely: In March 2006, FMCSA plans to complete an evaluation of the effectiveness of the ongoing Share the Road Safely pilot program in Washington state that FMCSA is jointly funding with the National Highway Traffic Safety Administration. The pilot combines education and enforcement to improve the safety of car drivers' driving behavior in the vicinity of trucks. If the results of the evaluation are favorable, FMCSA intends to extend the program to other states. * Border safety programs: FMCSA plans to conduct an evaluation of its border safety audit and compliance review programs in 2007. Border safety audits and compliance reviews can result in revocation of a motor carrier's authority to operate in the U.S., and FMCSA plans for its evaluation to cover this enforcement aspect of the program. In addition to its ongoing and planned program evaluations, FMCSA plans to develop a tool to help the agency set priorities for its enforcement programs by comparing their cost-effectiveness. The tool would produce estimates of cost-effectiveness by combining information from the models that FMCSA currently uses to estimate the impact of compliance reviews, roadside inspections, and traffic enforcement in terms of reductions in crashes, injuries, and fatalities with information on the costs of these programs. We have reported that considering information on both the costs and effectiveness of programs gives agencies a basis for focusing their efforts and improving their performance,[Footnote 38] and the Office of Management and Budget has guidelines for considering both the costs and benefits when making decisions to initiate, renew, or expand programs which would result in a series of measurable benefits or costs extending for three or more years into the future.[Footnote 39] FMCSA officials told us that the tool could also cover new entrant safety audits and certain education and outreach programs. As part of a separate effort, FMCSA has developed preliminary estimates of the average cost of a compliance review and the average cost of a roadside inspection. However, the estimates are not based on consistent assumptions, and an FMCSA official told us that the agency intends to reassess the costs for purposes of its prioritization tool.[Footnote 40] As recommended by the Office of Management and Budget, FMCSA also plans to start evaluating the costs incurred and the benefits realized by its existing regulations, beginning with evaluations of two regulations in 2006. Finally, in 2006 FMCSA plans to identify long-term trends in the motor carrier industry and how these trends could affect the agency's ability to meet its strategic goals and objectives over the next 20 years. FMCSA intends to use the results of this effort to support its policy development and strategic planning. We are encouraged by FMCSA's ongoing and planned efforts to obtain additional information on crash risk factors and on the costs and effectiveness of its enforcement programs because these efforts address long-standing core deficiencies that we identified in 1999 and 2000.[Footnote 41] We believe that the efforts will, if effectively implemented, provide FMCSA with a sound basis to refine and set priorities for its enforcement programs. MCSAP Is Designed to Improve Safety but Program Oversight Is Inadequate: MCSAP employs a performance-based approach to truck safety by encouraging states to analyze data to identify safety problems and target their grant activities to achieve reductions in truck crashes, injuries, and fatalities. However, FMCSA has not adequately overseen the development of states' safety plans. For example, FMCSA's grant planning meetings, in which it communicates priorities and how to develop plans with quantifiable goals, were conducted for less than a third of the states in fiscal year 2004, and FMCSA division offices that work with states did not sufficiently monitor and ensure states' progress towards safety goals. As a result, we were unable to determine whether states substantially met many of their safety goals. Furthermore, FMCSA has not completed its various oversight reviews of MCSAP in the past 3 years, as required by agency policy, and which, if carried out, could have helped to identify problems both with how states administer and the agency oversees the grant program. The agency, though, has recently taken steps to improve its grantee review program and strengthen its oversight of MCSAP. MCSAP Employs a Performance-Based Approach to Improve Safety: Since 1997, FMCSA has employed a performance-based approach for funding state commercial vehicle safety activities through MCSAP. The performance-based approach entails states analyzing data to identify commercial vehicle safety problems in their respective jurisdictions and targeting their grant activities to achieve FMCSA's primary goal of reducing truck crashes, injuries, and fatalities. As part of this approach, states must also demonstrate how their proposed goals and activities further other FMCSA goals, such as improving data quality and timeliness. In their safety plans, states are required to measure and evaluate their performance, which fosters accountability.[Footnote 42] Since 2000, FMCSA has convened several meetings to assist states in developing their plans. First, every spring, each service center is expected to hold a planning meeting attended by the state representatives and division office staff that administer and oversee the grant program. Service center and headquarters staff are to communicate the priorities for and changes made to the grant program for the coming year. The states are to reflect these priorities and changes in their safety plans. In addition, the Volpe National Transportation Systems Center is to provide states with training at planning meetings on analyzing data and developing goals, activities, and performance measures to address safety problems. Some states, such as New York and North Carolina, have also partnered with universities to conduct additional research and analysis on commercial vehicle safety.[Footnote 43] Finally, FMCSA's National Training Center conducts a MCSAP grants management training course that addresses, among other topics, how to develop safety plans, including goals, activities, and performance measures. Second, FMSCA conducts two examinations of the states' draft safety plans to ensure completeness, and that goals and activities are measurable and support FMCSA's mission. Initially, division office staff are to use a standardized checklist to ensure that states' safety plans incorporate the required elements. These elements include an identification of safety problems based on data analysis; goals, performance measures and activities that address state and agency priorities; evaluations of state progress towards the previous year's goals and activities; and certification of compatibility of state's motor carrier safety regulations with federal regulations. If any of the required elements are missing or unclear, the division office staff are expected to return the safety plan to the state for revision. FMCSA officials explained that after the division office staff approves the safety plan, they are to forward it for review by a "technical review panel" comprised of several service center and division office staff in that region before it is finalized and forwarded to headquarters. FMCSA staff at division offices, service centers, and headquarters consider these planning meetings and safety plan examinations to be valuable components of the grant program. We did not analyze the degree to which these meetings and safety plan examinations improved the states' safety plans. To further encourage improvements in commercial vehicle safety, FMCSA awards MCSAP incentive funds over and above the basic program funds to states that meet agency-specified safety criteria,[Footnote 44] such as reducing the number of commercial-vehicle-involved fatal accidents or the commercial vehicle fatal accident rate. States can also receive incentive funds if they meet other agency-specified safety performance criteria, such as the timely uploading of accident or inspection data, or checking the status of commercial driver licenses during all roadside inspections.[Footnote 45] FMCSA's Inadequate Planning and Oversight of Its State Grant Program Resulted in Uncertainty About Whether States Met Many of Their Safety Goals: Although MCSAP employs a performance-based approach to improving safety, it remains unclear whether the states we reviewed substantially met a majority of their safety goals. Using safety plans, quarterly reports, evaluations of previous years' efforts, and other monitoring information on states' activities, we assessed the extent to which seven state grantees substantially met their safety goals in fiscal year 2004. Of the 61 goals in states' safety plans, we could not determine whether 61 percent of these goals were substantially met.[Footnote 46] Of the remaining goals, we determined that 23 percent were substantially met, while 16 percent were not.[Footnote 47] We could not determine whether a majority of states' safety goals were substantially met because FMCSA did not thoroughly conduct three important oversight activities intended to help FMCSA make its own determination of states' progress towards their safety goals. First, FMCSA did not adequately carry out planning activities for 2004 MCSAP grants designed to ensure that states' safety plans included all of the key elements--quantifiable national and state goals, performance measures, and evaluations--needed to gauge whether state goals had been met. In this regard, planning meetings--where states receive assistance in developing goals, activities, and performance measures to address safety problems--were convened by only one of the four service centers. The meetings were not convened by three service centers because one did not have a state programs staff person to convene the meeting, another interpreted headquarters guidance to suggest that division offices work on MCSAP planning directly with states without service center assistance, and the third was constrained by states' budget restrictions at the time. Headquarters officials told us that they had expected planning meetings to occur that year. We also found that only one service center held technical review panels to ensure that safety plans included all of the key elements described above. One service center did not have a state programs staff person to convene a panel, while two service centers did not hold technical review panels because they believed their curbed oversight role of the service centers from fiscal years 1999 through 2003 implied that they should not conduct technical review panels. At the time, agency leadership restructured the agency to create a more direct relationship between headquarters and the division offices. Therefore, officials at these two service centers told us they felt that they did not have the authority to conduct many of their previous oversight activities. However, agency officials told us that they did not suspend technical review panels during that time period. Officials at the fourth service center--which originated the concept of the technical review panel--also acknowledged the curbed oversight role of service centers, but that they had conducted a panel for fiscal year 2004 because of their recognition that strong management should be maintained in their state grant programs. According to agency officials, current agency leadership, however, has encouraged service centers to assume a greater oversight role. Second, FMCSA division offices did not follow program guidance that requires ensuring that state safety plans incorporate key elements to adequately gauge states' performance. We found that goals or performance measures were not quantifiable, or were missing from various portions of the safety plans for the seven states we reviewed. When asked about this, staff from two of the division offices told us that quantifiable goals or performance measures in their safety plans are not always needed. However, we believe these key elements are critical because, without them, FMCSA would not be able to objectively assess whether states' goals had been substantially met. In contrast, staff from the other division offices with whom we spoke responded that they did not realize that multi-year goals should incorporate annual performance measures, or that goals and performance measures in the safety plans were not quantifiable or were missing. These staff agreed that incorporating such information into the safety plans would be appropriate and responded that they would consider doing so in the future. Finally, because FMCSA division offices did not adequately monitor states' progress towards safety plan goals, they did not obtain the information that would have enabled both them and us to determine whether state safety goals were substantially met. Offices monitor states' progress by reviewing quarterly reports, safety plan evaluations, and data reports, and through phone calls, E-mails and in- person meetings. For example, one of the seven states we examined--the sixth largest overall grantee in fiscal year 2004--did not submit required quarterly reports to its respective division office for several years. This was due to the responsible state staff person's belief that it was unnecessary to complete them. The division office followed-up with the state about this issue, but no further corrective action, such as withholding the state's future grant funding, was taken by the agency. The division office also reported that because the lead agency was in the process of changing, a new staff person would be assuming the responsibility and that they expect the quarterly reports to once again be submitted. In addition to the omission of quarterly reports, we found that three of the largest grantees did not include evaluations of both national and state goals in their safety plans, despite this being a program requirement. When we asked the respective division offices about this, they responded that this omission was not of concern because their office had a general sense of the states' progress through their monitoring. However, the information that these offices provided to support their monitoring effort, for the most part, did not allow us to assess whether the state had met its goals. Agency officials believe that some of the required information--such as quarterly progress reports and yearly evaluations--may no longer be necessary given their regular communication with states and access to information regarding state activities. However, given that MCSAP relies on a performance-based approach, the agency would not be able to adequately measure how grant funds are contributing to improving safety without this information. All of the division offices provided us with records of their monitoring, including data reports and E-mails. However, for the most part, their records did not provide enough information about states' progress for us to determine whether certain safety goals had been substantially met. Additionally, only one division office we reviewed formally tracks its state's activities by the goals specified in the safety plan, and this office has only recently begun to do so. Some division office staff conveyed their concerns regarding states' safety plans. The staff of two division offices recognized that their states' 2004 safety plans were not complete in terms of the key elements we discussed and that future improvements were needed. As a result, both division offices have explained that they are working more closely with the states to highlight these key elements in order to strengthen subsequent safety plans. Furthermore, the staff of all seven division offices generally understood the inability to gauge state performance based on the issues we identified and responded that they would consider our comments in the development of their states' 2006 safety plans. FMCSA Has Not Completed Various Oversight Reviews of Its Grant Program but Has Taken Steps to Improve Its State Review Program: FMCSA guidance states that three MCSAP-related reviews are to be conducted every 3 years: (1) division offices are to review their respective state grantees, (2) service centers are to review the division offices in their region, and (3) headquarters is to review the four service centers. Agency officials told us that the purpose of these reviews is to identify any weaknesses in the administration, monitoring, or oversight of the grant program by a state, division office, or service center. The reviews also identify best practices that could be applied to other states, division offices, or service centers.[Footnote 48] We found that FMCSA division offices reviewed only 19 of 56 grantees (34 percent) in the past 3 years. Of the completed reviews, issues cited include the incompatibility of various state motor carrier safety regulations with federal regulations, the need to better integrate performance-based approaches to state safety plans, missing quarterly reports and narratives that address a prior year's goals, and less-than- comprehensive annual evaluations of safety plan goals. Among the seven largest grantees, we followed up with the three division offices that had not completed reviews in the last 3 years to learn why they were not conducted. Two of these three division offices did not conduct reviews due to reorganization of the lead agency within their respective states. However, given that reviews tend to last between a week and several weeks and that there is a 3-year period to conduct state reviews, division offices would have had sufficient time to complete a review either before or after the reorganization. The third division office cited conducting a series of smaller "process" reviews as fulfilling the state review.[Footnote 49] When we asked headquarters about several division offices not conducting reviews of states, agency officials responded that they were disappointed many of the reviews had not been completed. However, they said they are taking steps--such as creating an implementation schedule of reviews to be conducted--to ensure that future state grantee reviews are completed in a timely fashion. The agency has also not reviewed division offices and service centers, as required by agency policy. In the past 3 years, the four service centers reviewed only 15 of 52 division offices (29 percent). We found that a contributing factor to the lack of reviews was the curbed oversight role of the service centers (as noted earlier) that was attributable to agency restructuring. The restructuring was designed to create a more direct relationship between headquarters and the division offices. We also learned that, among the four service centers, one did not conduct any division office reviews during the past 3 years because it believed that it should not conduct the reviews given the curbed oversight role of the service centers during part of that time period. Furthermore, headquarters has also not reviewed any of the service centers in over 3 years. Staff in headquarters cited a lack of staff and time as their primary reason for not completing these reviews. Headquarters staff devoted to MCSAP have steadily decreased over the past 11 years, from 11 full-time employees in 1995 to 4 full-time employees in 2005. In a majority of the 15 division office reviews conducted, service centers identified gaps in the monitoring and oversight responsibilities of division offices. Some of the recommendations from these reviews address commonly identified division office performance issues, including (1) not tracking, recording, or retaining information on their monitoring activities; (2) not providing consistent feedback to states about findings from monitoring activities; (3) not following- through to ensure improvements were made or tracking or recording any corrective actions that needed to be, or have been taken, by the state; and (4) not conducting grantee reviews as required by agency policy. Furthermore, two of the four service centers did not keep track of the division offices that have conducted state grantee reviews, and therefore they did not have the information to know which division office reviews had not been completed and what types of monitoring problems had been identified. Although we did not evaluate the extent to which division office reviews are necessary for the agency to conduct adequate oversight, we believe that the frequency with which monitoring problems were identified suggests the usefulness of these reviews. Lastly, we learned that headquarters does not keep track of whether division offices conduct state reviews. In addition to identifying gaps in monitoring and oversight, the agency also identifies what it considers to be best practices during its reviews of division offices. We have suggested that the sharing of best practices can be helpful in improving grantee performance.[Footnote 50] However, the agency has not fully shared those best practices among other division offices. Several best practices of division offices were identified during these reviews. Among them were (1) the development and use of an automated monitoring of state grant activities, (2) the convening of regular formal meetings with the state, (3) the development of a detailed quarterly report format that includes progress made towards safety plan goals, and (4) the hiring of an administrative grants manager. However, only two of the four service centers have actively circulated best practices from state reviews to division offices. The other two service centers told us that they intend to circulate these best practices in the future. In recognizing that the way in which states are reviewed has not been recently revised to reflect changes in MCSAP, and that it would like the reviews to have a greater focus on the financial aspects of the program, FMCSA has taken steps to improve its state grantee review program. In December 2004, the agency completed a draft plan for the new review program. The proposed program would be carried out by a joint team of division office, service center, headquarters, and contractor staff that would assist the division offices, and is comprised of three parts: a regulatory review, a financial review, and a performance review. The performance review would compare states' activities with the goals specified in their safety plans, and would emphasize the monitoring of performance. After a review is completed, a draft report of findings, recommendations, and other information would undergo review by the division office and the state. FMCSA plans for this program to follow a 4-year cycle for conducting the reviews. Agency officials told us that they have piloted this review program in four states, and officials expect the program will be fully implemented by the first quarter of fiscal year 2006. To ensure that this improved review program follows a 4-year cycle, FMCSA officials told us that they plan to schedule the reviews several years in advance and track which ones have been completed. Because this proposed review program is in the planning stage, we did not assess whether it is likely to address the problems we found. Further, FMCSA has several other efforts underway to strengthen the agency's oversight of MCSAP. First, agency officials noted that a newly created senior position to manage and oversee the service centers was recently filled, and that FMCSA plans to request three additional MCSAP staff in headquarters as part of its budget request for fiscal year 2007. Second, agency officials told us that they plan to incorporate an expectation of completing oversight reviews in the individual performance plans of all division office, service center, and headquarters staff that are responsible for MCSAP. Third, recent legislation requires that the agency report annually to Congress on the effectiveness of the grant program.[Footnote 51] Finally, FMCSA has two studies underway that relate to agency oversight of MCSAP: one to determine the effectiveness of division offices' activities, and the other to assure the quality of the agency's grant operations. Conclusions: In making decisions about its enforcement approach, FMCSA follows many of the effective management practices we identified related to performance management. We believe that it is noteworthy that FMCSA has begun to refine its enforcement approach based on information about the causes of crashes, that it plans to develop a tool to help it set priorities for its enforcement programs based on estimates of their cost-effectiveness, and that it has conducted--and continues to conduct--program evaluations of its enforcement programs, including evaluations that are assessing alternative approaches to enforcement. These efforts, if implemented effectively, should provide FMCSA with a reasonable basis for setting priorities for--and further refining--its enforcement programs in ways that reduce crashes, injuries, and fatalities in a cost-effective manner. However, because the agency's goals and measures for its enforcement programs do not describe expected and estimated contributions to FMCSA's overall goal of reducing the large-truck fatality rate, FMCSA and the public do not have a sense of how much the programs contribute to any reductions in the fatality rate that may occur over time. The absence of this kind of information makes it more difficult for FMCSA to make adjustments to its enforcement programs that could help achieve its fatality rate goal. In addition, because FMCSA does not know how much its civil penalties increase carriers' compliance with safety regulations, it lacks the information it needs to make sound decisions about any changes to its use of civil penalties; it also may be missing opportunities to increase carrier compliance, and ultimately safety, that could result from such changes. Furthermore, without targets specifying by how much it expects to reduce its backlog of enforcement cases and by when, Congress and other interested parties find it more difficult to hold the agency accountable for achieving this goal--one of the agency's four highest priorities for improving commercial motor vehicle safety in fiscal years 2005 and 2006. FMCSA's overall framework establishing performance and accountability for MCSAP is sound. It establishes expectations for safety improvements by grantees to help further the department's goal of saving lives and contains several mechanisms to help ensure that these expectations will be met. This is important, as MCSAP represents almost half of FMCSA's budget and the agency counts on it to materially contribute to saving lives. However, we found that the execution of these expectations has, too often, not been met, and aspects of how the agency actually works with states to oversee performance need strengthening. These conditions ranged from planning opportunities forgone, to differing perspectives about performance requirements on the part of division office and service center staff, to unfulfilled reviews of performance. While there are various reasons underlying the conditions that we found, all contributed to the end result of FMCSA not being in a position to determine whether states focused their attention on, and carried out, activities that could lead to lives saved. Further, the steps taken by the agency to strengthen safety plans and improve reviews of states performance are promising, but FMCSA should also assess its own oversight of the program. Recommendations for Executive Action: We are making five recommendations aimed at improving FMCSA's enforcement programs and MCSAP oversight: * To help ensure and demonstrate that FMCSA's enforcement programs contribute to the agency's goal to reduce the large truck fatality rate, we recommend that the Secretary of Transportation direct the FMCSA administrator to set goals for its enforcement programs that are designed to clearly demonstrate these contributions. * To improve FMCSA's ability to establish the most effective structure of civil penalties, we recommend that the Secretary of Transportation direct the FMCSA administrator to develop and implement one or more measures of the effectiveness of its civil penalties against motor carriers. * To improve the consistency and effectiveness of FMCSA's truck safety enforcement, we recommend that the Secretary of Transportation direct the FMCSA administrator to develop and include in its strategy and annual plans a specific numeric or measurable goal to eliminate the agency's backlog of enforcement cases, as required by the Motor Carrier Safety Improvement Act of 1999. * To improve accountability for use of MCSAP grant funds, we recommend that the Secretary of Transportation direct the FMCSA administrator to ensure that existing mechanisms, such as planning sessions, reviews of draft state plans, and periodic monitoring of states' progress, to the extent possible, are carried out and result in clarity of goals and performance measures and assessments of safety improvements made. * To improve FMCSA's oversight of MCSAP grantees, we recommend that the Secretary of Transportation direct the FMCSA administrator to (1) assess, upon implementation, whether the improved performance review of state activities are meeting the agency's intended goals; (2) incorporate MCSAP oversight as a segment of the effectiveness study of division offices; and (3) assess the oversight activities of service centers. Agency Comments and Our Evaluation: We provided a draft of this report to the Department of Transportation for review and comment and received comments from FMCSA officials. FMCSA generally agreed with the report's findings and agreed to consider our recommendations. FMCSA offered several corrections, which we incorporated in this report. We are sending copies of this report to congressional committees and subcommittees with responsibilities for commercial motor vehicle safety issues; the Secretary of Transportation; the Administrator, FMCSA; and the Director, Office of Management and Budget. We will also make copies available to others upon request. This report will be available at no charge on the GAO Web site at http://www.gao.gov. If you have any questions about this report, please contact me at (202) 512-2834 or [Hyperlink, siggerudk@gao.gov]. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff who made key contributions to this report are listed in appendix II. Signed by: Katherine Siggerud: Director, Physical Infrastructure Issues: [End of section] Appendixes: Appendix I: Scope and Methodology: In determining how the Federal Motor Carrier Safety Administration (FMCSA) used its enforcement authority to address noncompliance with the federal motor carrier safety regulations, we reviewed legislation, congressional reports leading to the creation of FMCSA, FMCSA regulations, manuals, guidance, and assessments of the agency's enforcement efforts by the Department of Transportation Office of Inspector General, and our own assessments. In addition, we obtained information from FMCSA about enforcement policies, the implementation of these policies and procedures, and analyzed enforcement data; we focused on fiscal years 1995 through 2004 in our efforts. We chose 1995 because it represented a period in which FMCSA partnered with industry and provided a 10-year period from which to observe changes in FMCSA's policies and actions. We discussed these requirements and the agency's implementation of them with FMCSA, industry and safety advocacy groups, and transportation researchers. We also analyzed FMCSA data on compliance reviews, roadside inspections, new entrant safety audits, homeland security-related visits of hazardous materials carriers, out-of-service orders and orders to cease operations, and civil penalties occurring between fiscal years 1995 and 2004. We used 2004 as the most recent year in our analysis because it is the most recent year for which complete data are available. In calculating the percentage of the motor carrier industry that receives compliance reviews, we used FMCSA's estimates of the annual size of the interstate motor carrier industry. This number is an estimate due to the fact that carriers are not required to report the closing of their operations to the department. This estimate has improved with the development of the new entrant audit program because FMCSA is able to identify non-interstate carriers and remove them from the total count. In determining total civil penalties assessed, we used the average civil penalty per violation, rather than the average civil penalty per enforcement case. We did this to observe how FMCSA treated different types of violations, such as recordkeeping and non- recordkeeping.[Footnote 52] We made several adjustments to FMCSA's data. First, we removed all motor coach cases from the dataset, because this report only focuses on large truck safety. During the past decade, passenger carriers have accounted for slightly more than 1 percent of enforcement actions. Second, we excluded all cases that the agency closed without enforcement or that remained open as of July 2005 (the end of our fieldwork). Between 1995 and 2004, an average of 145 cases was closed annually without enforcement, and an average of 19 cases was left open. FMCSA chooses to close a case without enforcement when the agency decides to cease seeking to collect a civil penalty. This can happen for several reasons, but frequently it is due to a carrier declaring bankruptcy. Excluding these cases ensured that the agency had completed the enforcement process before we analyzed the data. Since in many cases that are closed without enforcement the carrier is not formally charged with a civil penalty, the data tend to be skewed by including these cases. Third, we adjusted all civil penalties for inflation using 2004 dollars. In determining the reliability of FMCSA's data, we performed electronic testing for obvious errors in accuracy and completeness, and we interviewed officials from FMCSA's data analysis division who are knowledgeable about the data and how they were entered. FMCSA determined that the 1995 data on roadside inspections were not reliable. Due to this determination, we did not use these data. For the remainder of the data, we determined that the data were sufficiently reliable for the types of analysis we present in this report. In assessing the extent to which FMCSA follows key effective management practices in making decisions about its enforcement approach, we reviewed our reports on performance management, guidance from the Office of Management and Budget, and the Government Performance and Results Act to identify key effective management practices for agency decision making. The key practices that we identified were related to strategy, goals, performance measures, and decision making that considers the factors that contribute to large-truck crashes as well as the costs and effectiveness of agency programs. We obtained information on FMCSA's enforcement strategy, goals, and performance measures through discussions with FMCSA officials, and from FMCSA documents, including its budget request for fiscal year 2006 (which also serves as FMCSA's strategic plan, performance plan for fiscal year 2006, and performance report for fiscal year 2004). We also reviewed a report entitled Measuring the FMCSA's Safety Objectives from Year 2000 to 2002[Footnote 53] and reports on FMCSA's estimates of the impact of its enforcement programs. We obtained information on FMCSA's completed and ongoing crash causation studies and program evaluations, as well as how the results of some of these studies and evaluations have influenced FMCSA's decisions about refinements to its enforcement programs, through discussions with FMCSA officials or from FMCSA documents, including program evaluation plans and reports, and statements of work for planned evaluations. We also attended FMCSA's October 2004 public session in Falls Church, Virginia, designed to obtain input on its Comprehensive Safety Analysis 2010 initiative, which is intended to develop a new framework for overseeing truck safety. In assessing the extent to which FMCSA ensures that its Motor Carrier Safety Assistance Program (MCSAP) grants contribute to the agency's mission, we reviewed regulations and FMCSA guidance relating to the design of the grant program and discussed with FMCSA officials how accountability is built into it. We assessed FMCSA's planning and oversight of seven MCSAP state grantees in fiscal year 2004 (the latest full year for which information was available). These states-- California, Georgia, Illinois, New York, Ohio, Pennsylvania, and Texas- -comprised 27 percent of all MCSAP grants awarded for that year. Because we chose these states judgmentally (representing the largest grantees), we cannot project our findings nationwide.[Footnote 54] Reviewing a larger number of grantees would not have been practical due to resource constraints. To increase the reliability of our assessment, two analysts independently assessed whether states substantially met their goals. They then discussed their results and resolved any differences. We then discussed our assessment with FMCSA officials responsible for grant activities in those states. We also reviewed all four regional service centers to determine how they conduct their oversight activities. Finally, we attended a planning meeting for the fiscal year 2006 safety plans to gain a better understanding of how agency priorities are conveyed and how the agency provides assistance to the states in developing safety plans. [End of section] Appendix II: GAO Contact and Staff Acknowledgements: GAO Contact: Katherine Siggerud (202) 512-2834 or [Hyperlink, siggerudk@gao.gov]: Acknowledgements: In addition to the contact named above, James Ratzenberger, Assistant Director; Mallory Barg Bulman; Craig Fischer; David Goldstein; Tina Won Sherman; and Gregory Wilmoth made key contributions to this report. (542049): FOOTNOTES [1] Large trucks are those with a gross vehicle weight greater than 10,000 pounds. [2] FMCSA also has responsibility for improving the safety of commercial buses. Buses have a much better safety record than trucks; as a result, this report focuses on FMCSA's large truck enforcement activities. [3] For ease of reading, we use FMCSA to refer to the agency under all prior names. [4] The governor of the state designates a lead agency, such as the state police, to carry out grant activities, including developing the state's safety plans. [5] Based on data provided by FMCSA, we estimated that the economic impact resulting from crashes and incidents involving hazardous- materials vehicles was less than 5 percent of the total economic impact of truck crashes. Therefore, this report focuses on truck safety enforcement programs and not hazardous materials safety enforcement programs. [6] These information systems and program support FMCSA's enforcement activities and actions, such as targeting carriers for inspection. Because they are not enforcement activities, per se, we did not review their effectiveness. [7] In this report, we refer to all MCSAP grantees as "states." [8] These other safety initiatives include the Commercial Vehicle Information Systems and Networks program, PRISM program, and Intelligent Transportation Systems for Commercial Vehicle Operations program. MCSAP used to fund border grants. However, in fiscal year 2005, FMCSA created a separate program to fund border grants. [9] U.S. Department of Transportation, Office of Inspector General, Motor Carrier Safety Program, Federal Highway Administration, TR-1999- 091 (Washington, D.C. April 26, 1999). [10] GAO, Commercial Motor Vehicles: Effectiveness of Actions Being Taken to Improve Motor Carrier Safety is Unknown, GAO/RCED-00-189 (Washington, D.C. July 17, 2000). [11] In assessing FMCSA's actions, we used 1998 data as the benchmark for the partnering approach because this is the last year of data before the Department of Transportation's Inspector General released its report criticizing FMCSA. [12] Not all carriers are required to be reviewed every year. [13] Until 1998, state staff conducted all roadside inspections with partial support from federal funds. FMCSA has conducted an increasing number of inspections using its own staff; however, state staff still conduct more than 95 percent of all inspections. [14] We are assessing FMCSA's new entrant audit program as part of a separate study which we expect to issue shortly. [15] FMCSA uses a tool to help it determine the dollar amounts of its civil penalties. The tool uses nine criteria that consider the gravity of the violation and offense history of the carrier to develop recommendations for civil penalties. There have been statutory changes to this tool over the past decade. In 1998 and 2003, the agency adjusted all civil penalties for inflation. Further, in 1999, legislation created new categories of violations, which increased the allowable civil penalty for more serious violations to $10,000 and decreased the allowable civil penalty for record keeping violations. [16] FMCSA and state staff typically found more than one violation per enforcement case. The average penalty per enforcement case was approximately $3,700 higher than the average penalty per violation for each year. [17] Civil penalties for non-record keeping violations may be imposed up to $10,000 per violation. In 2004, the average civil penalty for these violations was slightly more than $2,000. [18] Violation types are not mutually exclusive; a violation can be both acute and non-record keeping. [19] FMCSA may reduce a civil penalty in cases where the motor carrier presents evidence to refute the violations found during the compliance review or offers mitigating circumstances to explain why the violations occurred. [20] This finding is not consistent with the 1999 report from the Department of Transportation's Inspector General. The Inspector General found that final assessments were substantially less than initial assessments from 1995 through 1998. This discrepancy may be due to different methodologies. Among other things, the Inspector General looked at a sample of cases, whereas we reviewed all enforcement cases. [21] Prior to this pilot program, FMCSA's efforts to improve the driving behavior of passenger vehicle drivers in the vicinity of trucks, which it initiated in 1994, were solely educational in nature; they did not involve an enforcement component. See GAO, Truck Safety: Share the Road Safely Program Needs Better Evaluation of Its Initiatives, GAO-03-680 (Washington, D.C. May 30, 2003). In the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users, Congress required us to update this study by June 30, 2006. [22] In February 2004, the Department of Transportation's Inspector General reported that, among other data quality problems, incomplete crash reporting and late or incomplete roadside inspection reporting by states adversely affect the reliability of FMCSA's carrier rankings. See U.S. Department of Transportation, Office of Inspector General, Improvements Needed in the Motor Carrier Safety Status Measurement System, MH-2004-034 (Washington, D.C. Feb. 13, 2004). In October 2004, Oak Ridge National Laboratory reported that while FMCSA's targeting tool is about twice as effective as random selection in identifying high-risk carriers, about 90 percent of the carriers that the tool identifies as "high-risk" are identified due to random variations in the underlying data rather than because they are consistently high-risk carriers. See Oak Ridge National Laboratory, Review of the Motor Carrier Safety Status Measurement System (SafeStat) (Oak Ridge, Tenn. Oct. 2004). FMCSA has taken some steps to address concerns about its targeting tool. We recently reported on FMCSA's efforts to improve the quality of its crash data; see GAO, Highway Safety: Further Opportunities Exist to Improve Data on Crashes Involving Commercial Motor Vehicles, GAO-06-102 (Washington, D.C. Nov. 18, 2005). [23] We have suggested that regulatory programs develop logic models to better understand how their programs deliver results in order to select appropriate goals. See GAO, Managing for Results: Strengthening Regulatory Agencies' Performance Management Practices, GAO/GGD-00-10 (Washington, D.C. Oct. 28, 1999). [24] See GAO, Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C. Feb. 26, 1999). [25] FMCSA considers a case closed when the final penalty amount has been determined. [26] The backlog in headquarters consisted of cases that had remained open for 6 months or more. FMCSA's regional offices also had--and the service centers that replaced them continue to have--their own backlogs of cases, but FMCSA did not track the size of these backlogs in 1999 and 2000. [27] Civil penalties comprised 81 percent of FMCSA's enforcement actions against motor carriers following compliance reviews during fiscal years 2002 through 2004. FMCSA considers compliance reviews to be its key enforcement program. [28] In fiscal year 2004, 46 percent of FMCSA's compliance reviews resulted in assessment of a civil penalty. [29] We did not evaluate the quality of these models because we are not using estimates produced by the model as a basis for our findings. [30] Although FMCSA conducted over 12,000 compliance reviews in 2002, only 9,172 of these reviews met the criteria for inclusion in FMCSA's model. [31] GAO, Executive Guide: Effectively Implementing the Government Performance and Results Act, GAO/GGD-96-118 (Washington, D.C. June 1996). We have reported that because outcomes, such as reduced fatality rates, are the result of the interplay of several factors including agencies' programs and factors external to agencies' efforts, it can be difficult and expensive to rigorously evaluate the relative contribution of multiple programs to the accomplishment of agencywide goals. See GAO, Managing for Results: Analytic Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington, D.C. May 30, 1997). [32] The total decrease of 0.14 does not equal the difference between 2.45 and 2.30 due to rounding. [33] We did not evaluate the quality of these studies because (1) we are not using data or findings from the studies as a basis for our findings; and (2) we are not evaluating whether the study results that FMCSA is using to inform its decision-making are based on reliable data, sound methodology, and appropriate analyses. [34] We did not evaluate the quality of the committee's review or FMCSA's changes for the same reasons that we did not evaluate the quality of any of FMCSA's studies, as described above. [35] FMCSA intends to achieve this shift by conducting more inspections that cover only drivers, more inspections that cover drivers and involve a walk-around check of the truck, and fewer of the more time- consuming inspections that cover drivers and involve a full inspection of the truck. [36] Safety groups have questioned the findings of these studies, while a major truck industry group agrees with the findings. As discussed above, we did not evaluate the quality of the studies. [37] GAO/GGD-96-118. [38] GAO/GGD-96-118. [39] Office of Management and Budget, Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs, Office of Management and Budget Circular A-94 (Washington, D.C. Oct. 29, 1992). [40] The cost estimate for roadside inspections includes fringe benefits and overhead associated with staff, whereas the cost estimate for compliance reviews does not include these costs. [41] GAO, Truck Safety: Motor Carriers Office Hampered by Limited Information on Causes of Crashes and Other Data Problems, GAO/RCED-99- 182 (Washington, D.C. June 29, 1999) and GAO/RCED-00-189. [42] For more information on how management's use of performance information can improve accountability, see GAO, Managing For Results: Enhancing Agency Use of Performance Information for Management Decision Making, GAO-05-927 (Washington, D.C. Sept. 9, 2005). [43] New York state partners with the State University of New York, Albany, and North Carolina has a partnership with the University of North Carolina, Chapel Hill. [44] In 1995, we reported that financial incentives in a grant program can spur performance by being tied to states' progress towards specified national goals. See GAO, Block Grants: Issues in Designing Accountability Provisions, GAO/AIMD-95-226 (Washington, D.C. Sept. 1, 1995). [45] In fiscal year 2004--the year for which we reviewed MCSAP oversight--Congress appropriated approximately $11 million for incentive funds. [46] For the purposes of this report, we considered that goals were substantially met if the state achieved 95 percent of its proposed goal. [47] In the instances where safety goals were not substantially met, division offices reported that states generally provided FMCSA with reasonable explanations as to why and that these explanations were usually beyond the control of the state. For example, one state set a target for its safety investigators to complete 642 compliance reviews in 2004. This target was set prior to the implementation of a legislatively-required new entrant safety audit program for which states had to shift resources from conducting compliance reviews towards conducting new entrant audits. Therefore, the state was able to complete only 348 reviews that year. Other explanations included a change in funding priorities for the state at the gubernatorial level and a delay in MCSAP funding that limited the amount of available state staff and time. [48] The reviews of states focus only on MCSAP and are more comprehensive in scope than the safety plan development and program monitoring activities carried out by the agency. [49] A process review focuses on one of the six review elements that are covered by the larger state review. While a minimum number of process reviews are not required to be fulfilled within a certain time period, program reviews are required to cover at least a 12-month period and each of the six review elements. In response to several states not conducting a program review because they had completed process reviews, agency officials agreed that the MCSAP guidance was ambiguous regarding this matter and that they would seek to clarify it as they update this guidance. They expect to complete updating the MCSAP guidance by the end of 2005. [50] See GAO, Student Mentoring Programs: Education's Monitoring and Information Sharing Could be Improved, GAO-04-581 (Washington, D.C. June 25, 2004). [51] Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users. [52] The average civil penalty per enforcement case is about $3,700 higher than the average civil penalty per violation, because each enforcement case may include several violations. [53] FMCSA, Measuring the FMCSA's Safety Objectives from Year 2000 to 2002, FMCSA-RI-03-018 (Washington, D.C. July 2003). [54] Results from nonprobability samples cannot be used to make inferences about a population, because in a nonprobability sample some elements of the population being studied have no chance or an unknown chance of being selected as part of the sample. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: