This is the accessible text file for GAO report number GAO-11-217R entitled 'Rail Transit: Reliability of FTA's Rail Accident Database' which was released on January 31, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-11-217R: United States Government Accountability Office: Washington, DC 20548: January 31, 2011: The Honorable Richard C. Shelby: United States Senate: Subject: Rail Transit: Reliability of FTA's Rail Accident Database: Dear Senator Shelby, As you requested, we conducted a review of challenges associated with enhancing safety on major rail transit systems.[Footnote 1]During the course of that review, we assessed the quality of data that the Federal Transit Administration (FTA) collects and maintains in its State Safety Oversight (SSO) Rail Accident Database. FTA, an agency within the Department of Transportation (DOT), collects these safety data, including data on types of accidents and causes, from SSO agencies and the rail transit agencies they oversee.[Footnote 2] FTA used the SSO Rail Accident Database to produce the agency's 2009 Rail Safety Statistics Report, which analyzed data from 2003 through 2008. [Footnote 3] Although we originally intended to report on safety trends using the SSO Rail Accident Database in the rail transit report, we determined that these data were not sufficiently reliable for such a purpose. As a result, in this review we further assess the SSO Rail Accident Database and FTA's processes for collecting and compiling the data. We determined that there are numerous inaccuracies in FTA's SSO Rail Accident Database and, consequently, the 2009 Rail Safety Statistics Report. FTA implemented changes to the data collection process over the past few years to address some of these issues. For example, FTA now requires SSO agencies to report incident specific information instead of aggregated or summary totals. In addition, FTA revised the definition of rail grade crossings to ensure consistent accident reporting by rail transit agencies, and now requests SSO agencies to provide unique incident tracking numbers to assist with data validation and prevention of duplicate entries. FTA is also working to validate and correct existing data in the SSO Rail Accident Database. Despite these changes, the SSO Rail Accident Database and the 2009 Rail Safety Statistics Report still include erroneous data. DOT is seeking legislative authority to directly regulate and enforce rail transit safety, in part on the basis of recent trends in accidents and injuries. There were several legislative proposals introduced during the 111th Congress to give FTA authority to establish safety regulations for rail transit agencies and, in cooperation with the states, oversee and enforce compliance by these rail transit agencies with these regulations.[Footnote 4] If FTA assumes greater oversight authority over rail transit agencies, accurate and reliable rail safety data will be critical. The purpose of this report is to convey our findings about how insufficient internal control over the entry and processing of rail transit safety data has negatively affected the reliability of the data contained in FTA's SSO Rail Accident Database and its 2009 Rail Safety Statistics Report. Summary of Results: Our analysis of the reliability of data contained in the SSO Rail Accident Database found data discrepancies, such as unverified and duplicative entries, as well as insufficient internal control. Consequently, FTA's 2009 Rail Safety Statistics Report, which focuses attention on safety issues in the public rail transit industry, and is intended to support FTA's mission to identify safety priorities and strategies to address industry concerns, includes unreliable data. This affects the accuracy of the statistics regarding rail transit safety, including those used for producing information on trends over time, which can be safety indicators to help guide FTA's safety oversight efforts. For example, data discrepancies may under or overstate accidents, injuries, fatalities, and property damage totals, among other things. FTA has implemented some changes to the data collection process and has begun to validate data and correct discrepancies contained in the SSO Rail Accident Database. However, FTA's validation proposal does not contain specific efforts to establish procedures that would improve data reporting in the future. We are making recommendations to DOT to help FTA improve internal control over the process used to obtain data from SSO agencies and ensure the accuracy and reliability of the SSO Rail Accident Database. Scope and Methodology: The information in this report is based on work conducted from September 2010 to January 2011, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In conducting our work, we interviewed FTA officials and analyzed SSO and rail transit agency data to assess the extent to which rail transit safety data used for FTA's rail safety statistics report were complete and accurate and avoided unverified and omitted data entries, duplicative entries, and other data discrepancies. For our engagement, we focused primarily on the SSO templates and FTA's SSO Rail Accident Database since these are used by FTA to produce the rail safety statistics report. In order to ensure we were assessing the exact data FTA uses, we replicated the first five summary tables presented in that report. We reviewed SSO data from 2003 through 2008 and assessed the extent to which FTA reconciles the data reported by SSO and rail transit agencies with the National Transit Database (NTD) and other supplemental sources such as the National Transportation Safety Board (NTSB), and how the FTA's procedures contribute to the reliability of the data in its rail safety statistics report. To determine the extent to which FTA's internal control over the entry and compiling of rail transit safety data influences the reliability of the data and annual reported statistics, we reviewed FTA's rail transit safety data collection policies and procedures for SSO agencies and rail transit agencies and identified examples of data discrepancies such as unverified and duplicate entries. We assessed FTA's processes against GAO's Standards for Internal Control in the Federal Government. Background: FTA relies on two primary information sources for its SSO Rail Accident Database: SSO agency data and the NTD. SSO agencies are required annually to submit data to FTA on rail transit accident and hazard investigations including date, type of accident, number of injuries, number of fatalities, probable cause, property damages, and type of individuals injured, such as passenger or worker. These annual data submissions are manually entered into a template that the SSO agencies submit to FTA. FTA recommends, but does not require, SSO agencies to provide internal tracking numbers assigned to each accident, but not all SSO agencies do so. For example, in 2008, SSO agencies for North Carolina, Illinois, Louisiana, and Utah did not use internal tracking numbers to record their accidents. The NTD is a separate system that includes information on rail safety reporting, as well as operating statistics. Transit agencies are required to provide data monthly to the NTD through an Internet-based reporting system. NTD data include more detailed information compared with SSO agency reports, such as the incident time, location, and descriptions, but it does not contain probable cause determinations. NTD data also have an incident number unique to NTD that does not correspond to SSO agencies' tracking numbers. According to FTA officials, FTA's Office of Safety and Security relies heavily on contractors to compile information primarily from SSO agencies and the NTD system into the SSO Rail Accident Database. This information may be supplemented with data from the National Highway Traffic Safety Administration (NHTSA), the Federal Railroad Administration (FRA), the NTSB, and the National Safety Council (NSC). FTA used the SSO Rail Accident Database to produce the agency's 2009 Rail Safety Statistics Report, the most current safety report available. FTA's Lack of Internal Control Contributes to Unreliable Rail Transit Safety Data: The SSO Rail Accident Database contains data discrepancies such as unverified and duplicative entries, rendering the data unreliable. Although we have not fully assessed the extent of reporting errors, we have found sufficient problems with internal control to warrant our concern about the reliability of FTA's SSO Rail Accident Database and reports produced from these data. According to GAO's standards for internal control in the federal government, internal control is a major part of managing an organization and comprises the plans, methods, and procedures used to meet missions, goals, and objectives. Internal control supports performance-based management and also serves as the first line of defense in safeguarding assets and preventing and detecting errors and fraud. In short, internal control, which is synonymous with management control, helps government program managers achieve desired results through effective stewardship of public resources. Control activities specific for information systems that would help ensure completeness and accuracy of FTA's SSO Rail Accident Database include (1) all authorized transactions entered into and processed by the computer; (2) reconciliations performed to verify data completeness; (3) the agency's data entry design features contribute to data accuracy; (4) data validation and editing performed to identify erroneous data; and (5) erroneous data captured, reported, investigated, and promptly corrected. The following examples illustrate instances where a lack of these control activities has led to unreliable or inaccurate data contained in FTA's SSO Rail Accident Database: * Unverified data entries and other discrepancies: Unverified data entries can affect data accuracy. We found that FTA's SSO Rail Accident Database contains a number of unverified data entries. Unverified accident reporting may under or overstate accidents, injuries, fatalities, and property damage totals, among other things, depending on the nature of the discrepancy. - There were 84 entries in FTA's SSO Rail Accident Database for years 2003, 2004, 2005 that had an accident date of January 1. For example, FTA's SSO Rail Accident Database lists 34 incidents occurring on January 1, 2003, for Maryland Transit Administration (MTA). FTA officials stated that these entries occurred because SSO agency data was collected and certified in aggregate (i.e., SSO agencies only provided summary accident totals) for the year. Beginning in 2006, FTA required SSO agencies to provide incident specific information that could be more easily validated for accuracy. However, the use of January 1 for multiple accidents that did not actually occur on that date makes it impossible to verify whether these accidents were appropriately included in the rail transit safety report's statistical and trend analyses for 2003, 2004, and 2005. - Of the 3,666 entries for 2003 through 2008 in FTA's SSO Rail Accident Database, 818 (22 percent) were entries from the Southeastern Pennsylvania Transportation Authority (SEPTA), the fourth largest light rail transit system in the country. This was more than nine times the 87 entries for Massachusetts Bay Transportation Authority, the fifth largest light rail transit system in the country. FTA officials told us that SEPTA reported rail grade crossing accidents inconsistently with other rail transit agencies from 2003 through 2005. For example, SEPTA was including collisions occurring at intersections of driveways and alleyways and other midblock locations. The effect of this inconsistent reporting is that the entire number of reportable accidents included in the SSO Rail Accident Database from 2003 through 2005 is overstated, which distorts accident rate trends in the 2009 Rail Safety Statistics Report. Once FTA identified this issue, the agency tightened its reporting definition and provided technical assistance on the new definition to obtain more consistency in how incidents are reported. As a result, SEPTA began reporting accidents in a more consistent manner from 2006 forward. To address the inconsistency of rail grade crossing collisions between 2003 and 2005, FTA is working with SEPTA to review the data. - Of the 3,666 accident entries from 2003 through 2008 in FTA's SSO Rail Accident Database, 82 percent did not have a tracking number. Between 2003 and 2005, agencies did not provide tracking numbers because, as discussed above, data were reported in aggregate during that time period. We observed that from 2006 through 2008, many SSO agencies included tracking numbers in their accident reports to FTA during those years. However, FTA's SSO Rail Accident Database did not contain any tracking numbers prior to 2008 even though these data were submitted to the agency and could be used for internal control. The use of unique tracking numbers as assigned by each SSO agency would assist FTA in verifying the rail safety data and prevent duplication of entries. - The SSO Rail Accident Database indicates that an accident occurred on the New York City Transit (NYCT) system on April 29, 2007. According to the database, there was one worker fatality, no other workers injured, and five passengers were injured. FTA officials stated that they verified the worker fatality and added the five injured passengers from the NTD report of this accident. However, according to a NYCT senior official, there was in fact one worker killed, one worker injured, but no passengers reported injured in the accident. Thus FTA's SSO Rail Accident Database may not include one worker injured and may overstate five passenger injuries. - SSO accident thresholds require that accidents must be reported if they result in "injuries requiring immediate medical attention away from the scene for two or more individuals." Below are two examples of accidents in the SSO Rail Accident Database that reflect inaccurate reporting. FTA officials maintain these are accurately reported injuries. We disagree. The inflated quantity of injuries from these two examples remain in the SSO Rail Accident Database and were used to compile statistics in the 2009 Rail Safety Statistics Report. -- A Chicago Transit Authority (CTA) train, number 220, derailed between Clark/Lake and Grand/Milwaukee Stations in Chicago, Illinois. FTA's SSO Rail Accident Database lists a date of July 10, 2006, for this accident, with injuries to 257 passengers and four workers. NTSB, as mentioned above, is a source FTA uses to verify data. However, the NTSB report for this accident states that "152 persons were treated and transported from the scene, including three injured firefighters and 1 injured CTA supervisor." In addition, the NTSB report has an accident date of July 11, 2006. FTA officials maintain their injury count is correct stating that NTSB reports are often not finalized until a year or two after the accident. FTA officials also maintained that the additional 105 injuries occurred to passengers who claimed they received immediate medical attention away from the scene in the days following the accident. FTA officials acknowledge that CTA has identified approximately 40 of the 105 people who claimed immediate injuries in the derailment were not on the train; nevertheless these injuries are still included in the database. -- In the SSO Rail Accident Database, NYCT reported 1 track worker fatality and 19 injured track workers resulting from a collision occurring on April 24, 2007. However, the official accident report of this accident states that 1 track worker was killed during the collision and does not mention any injured workers. Upon contacting NYCT, we were able to confirm with the agency that, while they reported 19 injured workers, these workers were neither injured at the scene nor immediately transported for medical attention, but rather suffered trauma from witnessing the accident in the days following. This error is significant and resulted in overstating overall injury totals, as well as injured worker totals, in the 2009 Rail Safety Statistics Report by 19, or about 12 percent of all injured workers in the rail safety database for 2003 through 2008. * Duplicative entries: Duplications in data entry overstate injuries and property damage totals, among other things. In our review of SSO agency data, our analysis found 76 of 3,666 records (or about 2 percent) that were duplicative entries (i.e., the incident was entered twice). - FTA agreed that 60 of these duplicative entries were a result of three agencies submitting duplicate incident data reports to FTA. These errors were identified by FTA in 2009, and changes were made that same year to the incident update submission process to address this flaw. However, FTA used the inflated quantity of duplicate accidents from these three agencies when compiling statistics for the 2009 Rail Safety Statistics Report. -- FTA officials stated that about 25 incidents in FTA's SSO Rail Accident Database for the calendar year 2008 were entered twice--once under "Denver RTD," or Regional Transportation District, and again under "RTD." FTA confirmed this error was not caught until after the publication of the 2009 Rail Safety Statistics Report. Each duplicate incident, and all related data associated with that duplicate record, inflates accident rates and trends. For example, the overstatement of property damage from this 2008 error was $251,249, and passenger injuries were overstated by six. -- In February and March of 2007, New Jersey's Department of Transportation reported multiple incidents on the same day for the Port Authority Transit Corporation (PATCO). FTA officials agreed these entries for PATCO were entered twice. -- In July 2007, Washington State's Department of Transportation reported multiple incidents on the same day for Sound Transit-Tacoma Link. FTA officials agreed these entries for Sound Transit-Tacoma Link were entered twice. - We found 16 additional examples of duplicate entries containing identical or near identical information. FTA stated that these 16 entries to the database were reported by the transit agencies as separate incidents, and therefore, FTA does not consider them duplicate entries. We disagree and believe these records are erroneously included twice in the SSO Rail Accident Database, thus inflating statistics in the 2009 Rail Safety Statistics Report. Examples of these include: -- Two SEPTA entries for September 15, 2004, that include an injury are in the database. When we compared the descriptions, one entry reported that "an 81-year-old female pedestrian was struck by an eastbound Route 36 trolley at Island Ave. and Lindbergh Blvd." and that "two shuttle buses" were subsequently requested for stranded passengers. The second entry states "an elderly female walked in front of trolley" and "two shuttle buses" were requested. -- Two entries reporting an injury on the Massachusetts Bay Transportation Authority (MBTA) are in the database on September 30, 2005. FTA officials stated that they appear to be two perhaps related, but separate, events with different railcars impounded, and that MBTA reported the two incidents separately. However, when we compared the descriptions of these two entries, both reported an "unidentified white male with lacerations" and referenced the same impound number. -- Two entries are in the database reporting a pedestrian struck by a San Diego Trolley train, resulting in a fatality on May 2, 2005. FTA officials stated that two separate reports were submitted for this event, and it was not a duplicate entry. However, the location, car number involved, and property damage total contained in these two records are all identical. In addition, the incident description is nearly identical describing a male pedestrian on an "elevated concrete structure" who lost his footing and fell into the path of the train. FTA has identified the cause of some of the data inaccuracies and made some changes to reporting requirements, such as defining rail grade crossings more consistently, and requiring SSO agencies to report incident specific information instead of aggregated or summary totals. However, FTA needs to establish appropriate information system control activities, such as those mentioned earlier in this report, in order to improve data accuracy and completeness in any future data collection efforts. For example, if FTA's SSO agency reporting process included data entry design features that prevented unverified or duplicative entry and required certain fields to be completed, such as incident tracking numbers, locations, and descriptions, some of the data discrepancies listed above would likely not have occurred. FTA officials have acknowledged the important role that data play in making decisions to address challenges to rail transit safety. However, our analysis of the data reliability concerns we have identified--including data discrepancies such as unverified and duplicative entries--along with insufficient internal control, demonstrates the unreliability of FTA's rail transit safety data. Moreover, the SSO Rail Accident Database and the 2009 Rail Safety Statistics Report still include erroneous data. The lack of reliable data limits FTA's ability to produce accurate accident rates and trend information. As mentioned above, FTA officials acknowledged data inaccuracies and inconsistencies and have implemented changes to the data collection process over the past few years to address some of these issues. In addition, FTA is currently working to validate the SSO Rail Accident Database by conducting data comparisons with NTD and contacting rail transit agencies to identify and correct for discrepancies, as appropriate. However, the validation effort proposed by FTA to correct inaccuracies for previous years does not contain specific efforts to establish procedures that would improve data reporting in the future, such as internal control over SSO agency reporting templates that are submitted to FTA, to ensure completeness and accuracy. More complete and accurate data, and reporting by SSO agencies, would better enable FTA to identify safety priorities for rail transit, determine the effectiveness of its programs, and provide an accurate picture of overall transit safety. Recommendations for Executive Action: We recommend that the Secretary of Transportation direct the Administrator of FTA to take the following two actions: * Develop and implement appropriate internal control activities to ensure that the data entered into SSO agency reporting templates are accurate. To accomplish this, the Administrator should consider data entry design features to ensure consistency in reporting across rail transit agencies. * Incorporate appropriate internal control over the method used to review and reconcile SSO agency data with other data sources to better ensure accuracy and reliability of the SSO Rail Accident Database. Agency Comments and Our Evaluation: We provided a draft of this report to the Department of Transportation for comment. DOT agreed to consider the recommendations in this report and provided technical comments and clarifications, which we incorporated, as appropriate. We are sending copies of this report to the Chairs and Ranking Members of the Senate Governmental Affairs Committee and the House Government Reform Committee and other appropriate congressional committees, as well as to the Secretary of Transportation and the Administrator of the Federal Transit Administration. The report also is available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff members have any questions concerning this report, please contact me at (202) 512-2834 or wised@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report were Brandon Haller, Assistant Director; Maren McAvoy; Martha Chow; Antoine Clark; Kathleen Gilhooly; Judy Guilliams-Tapia; Hannah Laufe; Grant Mallie; and Anna Maria Ortiz. Sincerely yours, Signed by: David J. Wise: Director, Physical Infrastructure Issues: [End of section] Footnotes: [1] GAO, Rail Transit: FTA Programs are Helping to Address Transit Agencies' Safety Challenges, but Improved Performance Goals and Measures Could Better Focus Efforts, [hyperlink, http://www.gao.gov/products/GAO-11-199] (Washington, D.C.: Jan. 31, 2011). [2] Through the State Safety Oversight Program, FTA monitors 27 state agencies that oversee the safety and security of rail fixed guideway systems in 26 states. See 49 C.F.R. part 659. One state, Illinois, has two oversight agencies, each overseeing a different rail transit agency. [3] DOT, FTA, 2009 Rail Safety Statistics Report (Washington, D.C: 2010). [4] Public Transportation Safety Act of 2010, S. 3638, 111th Cong. (2010); Public Transportation Safety Program Act of 2010, S. 3015 and H.R. 4643, 111th Cong. (2010). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: