Aviation Safety: FAA's Risk-Based Oversight for Repair Stations Could Benefit from Additional Airline Data and Performance Metrics [Reissued on September 2, 2016]
Highlights
What GAO Found
The extent to which U.S. airlines contract out aircraft maintenance to domestic and foreign repair stations (as opposed to performing maintenance in-house) has remained relatively steady from 2010 through 2014. GAO's analysis of Bureau of Transportation Statistics (BTS) data for that period (the most recent available) showed that 28 selected U.S. airlines' annual contracted maintenance spending ranged from 58 to 64 percent of their total annual maintenance spending. In addition, representatives for all 10 U.S. airlines GAO interviewed said that the type of aircraft maintenance can affect an airline's decision to contract out maintenance. For instance, airlines generally indicated that the majority of light, routine maintenance--i.e., more suited for overnight or quick turnaround--is performed in-house. However, when it is more cost-effective, they contract some, if not all, of the more involved maintenance and repairs that may require specialized skills and equipment. Industry representatives GAO interviewed also identified three key influencing factors that affect airlines' maintenance decisions: (1) service quality available at repair stations, (2) cost considerations, and (3) the use of service contracts with manufacturers of original aircraft parts.
The Federal Aviation Administration's (FAA) regulatory requirements for foreign repair stations differ in several ways from the requirements for domestic repair stations, including certification, renewal, personnel, and drug and alcohol testing requirements. FAA faces challenges in overseeing foreign repair stations related to its inspectors' ability to conduct routine and unannounced inspections of foreign repair stations. FAA also faces challenges with coordinating inspections of repair stations conducted by its oversight offices for airlines and its oversight offices for repair stations, with some stations receiving around 50 visits per year from the airline oversight offices alone. In response, FAA is implementing new strategies aimed at more efficiently using its resources.
In fiscal year 2015, FAA began to deploy its Safety Assurance System (SAS), a risk-based, data-supported oversight system to help standardize how its inspectors identify safety risks in planning and conducting oversight, including of repair stations. Safety assurance is one of the four components of FAA's new Safety Management System oversight approach, and FAA designed SAS to implement that component. GAO found the design of SAS fully meets three of the five principles FAA identified as key for the safety assurance component and partially meets the other two principles, which involve data collection and management review. SAS enables inspectors to collect various data, but does not enable them to consider a repair station's volume of work when determining risk. FAA does not otherwise collect or track this data. FAA officials said they do not consider volume data to be a standalone risk factor, but FAA has previously stated that tracking volume data could help identify high-risk repair stations. GAO and the Department of Transportation's (DOT) Office of Inspector General have reported on the importance of FAA's collection of quality data for providing a comprehensive risk-based oversight system. Also, FAA conducts management reviews, but has not developed a process with goals and performance metrics for determining the effectiveness of SAS. Without the ability to measure progress toward goals, FAA risks not knowing whether its new, risk-based oversight approach is a success or could be improved.
Why GAO Did This Study
FAA is responsible for overseeing nearly 4,800 FAA-certificated repair stations in the United States and in foreign countries. While U.S. airlines' rely on repair stations for much of their maintenance, some aviation stakeholders have questioned FAA's oversight of foreign repair stations.
GAO was asked to examine maintenance contracting trends and FAA oversight of repair stations. This report assesses: (1) trends and factors influencing airline maintenance contracting from 2010 through 2014, (2) how FAA's oversight of foreign and domestic repair stations differs and associated challenges, and (3) efforts taken by FAA to improve its risk-based oversight. GAO analyzed BTS data on airlines' maintenance spending from 2010 through 2014 for 28 selected U.S. commercial airlines with the largest number of flights, and interviewed representatives for 10 U.S. airlines. GAO visited seven foreign repair stations in three countries, and interviewed industry representatives. GAO also analyzed FAA inspection and enforcement data for repair stations from 2010 through 2014.
Reissued on September 2, 2016
Recommendations
FAA should: (1) develop and implement a process for incorporating into SAS volume data for U.S. airlines’ maintenance contracted to repair stations and (2) develop a process to evaluate the effectiveness of SAS. FAA disagreed with the first recommendation, noting it considered other volume-related data, and agreed with the second recommendation. GAO continues to believe the recommendation is valid as discussed in this report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Transportation | To enhance FAA's risk-based approach for oversight of repair stations, the Secretary of Transportation should direct the Administrator of the Federal Aviation Administration to develop and implement a process in Flight Standards for incorporating into SAS the volume of critical maintenance that each U.S. airline contracts to repair stations. |
FAA continues to indicate that the agency does not concur with this recommendation and does not plan to take any actions to implement it because the agency believes the subjective nature of volume of work makes it an ineffective risk indicator. As of January 2024, FAA stated the agency monitors many risk indicators within the FAA's Safety Assurance System (SAS), such as high workforce turnover; changes in management; rapid growth or downsizing, changes in aircraft complexity/programs, financial conditions, and age of fleet. FAA noted that it uses these risk factors, among others, to assess repair stations to ensure they have the necessary facilities, equipment, materials, and personnel consistent with their ratings. FAA officials said they have made improvements to enhance SAS tools and resources that the agency believes further allows them to better assess risks at repair stations. Within SAS, for example, when a new hazard is identified, safety inspectors may add action items for follow up to address a safety issue. Further, FAA's process includes evaluating the new hazard and responding with an overall risk value. While volume data is not specifically collected, FAA contends the risk management process within SAS provides for the identification, analysis, and disposition of newly identified hazards. Without incorporating data on the volume of maintenance work performed for U.S. airlines into FAA's repair station oversight process, FAA's ability to administer a comprehensive risk-based oversight system could be limited. This information would help determine trends in airlines' use of contractors and identify the often-used repair stations, and without doing so, FAA risks degrading its oversight of airlines' maintenance programs and delaying the identification and prioritization of potential high-risk repair stations.
|
| Department of Transportation | To enhance FAA's risk-based approach for oversight of repair stations, the Secretary of Transportation should direct the Administrator of the Federal Aviation Administration to develop and implement an evaluative process with measurable performance goals and measures to determine the effectiveness of SAS as the SMS safety assurance component. |
In fiscal year 2015, FAA began deploying its Safety Assurance System (SAS), a risk-based, data-supported oversight system, for domestic and foreign repair stations-which U.S. airlines rely on for much of their aircraft maintenance work. SAS is designed to be the safety assurance component of FAA's new Safety Management System (SMS) oversight approach and fully meet SMS's five safety assurance principles. In 2016, GAO reported that SAS fully met three of the principles, but partially met the data collection and management reviews principles. While FAA collects various data on repair stations, FAA does not collect or track data on the volume of work U.S. airlines send to repair stations, which is needed to fully meet the data collection principle. FAA conducts management reviews, but has not established SAS goals or performance metrics to measure overall system progress, a step necessary to fully meet the management reviews principle. Furthermore, FAA management officials stated that there was no formal process for determining the effectiveness of SAS, including its effectiveness in improving the oversight of certificate holders (i.e., airline operators), including repair stations. While the agency was considering this element and was at the very beginning phase of this process, FAA had not developed specific performance goals and metrics for SAS, or developed a timeline for doing so. Therefore, GAO recommended that FAA develop and implement an evaluative process with measurable performance goals and measures to determine the effectiveness of SAS as the SMS safety assurance component. In 2020, FAA developed and implemented a process to evaluate the effectiveness of SAS. FAA updated its policy order related to how the agency manages risks associated with its oversight of certificate holders using SAS. For instance, the order states that considerations to adjust the risk indicators include, among other things: 1) controls and process measures for managing changes in operations and maintenance; 2) experience of personnel with new technologies and the operational environment; and 3) audits to assess the effectiveness of the certificate holder's training programs and operation within its environment. FAA's oversight offices will use available data and other inputs in the development of the Comprehensive Assessment Plan, a tool for planning and managing assessments. The offices uses the plan to add assessments and data collection tools, adjust the resource order, adjust due dates of assessments, and record the reasons for making adjustments. FAA will also use risk indicators in the Certificate Holder Assessment Tool to associate risk to the certificate holder's proposed or current operating system. In addition, the offices will use a risk-profile assessment tool to review the risk profile and determine if any adjustments to the Comprehensive Assessment Plan are necessary. As a result, FAA is in a better position to measure progress toward goals and determine the effectiveness of the agency's risk-based oversight approach.
|