Aviation Safety:

System Safety Approach Needs Further Integration into FAA's Oversight of Airlines

GAO-05-726: Published: Sep 28, 2005. Publicly Released: Oct 26, 2005.

Additional Materials:

Contact:

Gerald L. Dillingham
(202) 512-4803
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Federal Aviation Administration (FAA) uses the Air Transportation Oversight System (ATOS), which was developed around the principles of system safety, to oversee seven "legacy airlines" and nine other airlines. In this report, we refer to airlines that are not in ATOS as non-legacy airlines. Two other processes are used to oversee 99 non-legacy passenger airlines, which represent a fast-growing segment of the commercial aviation passenger industry and carried about 200 million passengers in 2004. The National Work Program Guidelines (NPG) establishes a set of inspection activities for non-legacy airlines. The Surveillance and Evaluation Program (SEP) uses principles of system safety to identify additional risk-based inspections for those airlines. GAO's objective was to assess the processes used by FAA to ensure the safety of non-legacy passenger airlines. GAO reviewed the strengths of FAA's inspection oversight for non-legacy passenger airlines and the issues that hinder its effectiveness.

A key strength of FAA's inspection oversight of non-legacy airlines is the introduction of system safety concepts to some inspections, which FAA accomplished by adding SEP to its traditional inspection process, NPG. Although NPG has risk-based elements, it lacks the structured approach to risk identification found in SEP. Under SEP, data are used to help determine trends or problems. The SEP process uses a team of inspectors to identify inspection activities, which we have previously reported is generally more effective than the use of individuals due to their collective ability to identify risks. Under SEP, inspectors also ascertain risks internal to FAA, such as staffing shortages. FAA's oversight of non-legacy airlines further incorporates processes to ensure that inspectors follow up on airline actions taken in response to inspection findings. These efforts address several past GAO concerns, including that NPG did not allow FAA to identify risks and allocate inspection resources accordingly. The full potential of FAA's inspection program for non-legacy airlines, however, is not being realized due to incomplete implementation of its system safety approach and other challenges. The inspection workload is still heavily oriented to nonrisk-based activities, with 77 percent of inspection activities being identified through the NPG and the remaining relatively small percentage identified through SEP. The emphasis on NPG, including FAA's guidance that inspectors must complete NPG-required inspection activities, acts as a disincentive to identifying further inspection activities through SEP. Inspectors face workload challenges as staff lost through attrition may not be replaced due to a hiring freeze. FAA estimates that over 1,100 inspectors of non-legacy airlines will leave the agency in fiscal years 2005 to 2010. In addition, some FAA inspectors indicated that a lack of technical training on airline systems and equipment posed potential risks to the agency's oversight process. Finally, FAA lacks a process to communicate information to inspectors on how certain internal risks identified through SEP are being resolved. Moreover, FAA has not established a process to evaluate the effectiveness of SEP.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To improve the effectiveness of the agency's oversight of non-legacy airlines, the Secretary of Transportation should direct the FAA Administrator, To improve the effectiveness of the agency's oversight of non-legacy airlines, the Secretary of Transportation should direct the FAA Administrator, in order to ensure that all regional and field offices have a complete and timely understanding of FAA's policies relating to the inspection process, to improve training in areas such as risk management, coding items in the PTRS database, and how and under what circumstances SEP-identified activities can replace NPG-identified activities through retargeting.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: FAA revised the "National Flight Standards Work Program Guidelines (Order 1800.56F) to provide detailed instructions to regional and field offices concerning the inspection policy and process and data entry, coding, and data quality related to documenting inspections. The order also provides procedures for retargeting inspections. The SEP training course for inspectors was revised to include these program changes.

    Recommendation: To improve the effectiveness of the agency's oversight of non-legacy airlines, the Secretary of Transportation should direct the FAA Administrator, in order to ensure that all regional and field offices have a complete and timely understanding of FAA's policies relating to the inspection process, to improve communication in areas such as whether and how internal risks identified by inspectors have been resolved.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: FAA revised SEP program guidance to require a separate entry in the inspection database to track each internal risk from identification to resolution and closure. FAA also revised the "National Flight Standards Work Program Guidelines" (Order 1800.56F) to provide detailed instructions to regional and field offices concerning the inspection policy and process and data entry, coding, and data quality related to documenting inspections.

    Recommendation: To improve the effectiveness of the agency's oversight of non-legacy airlines, the Secretary of Transportation should direct the FAA Administrator, to improve its safety oversight of airlines, to develop a continuous evaluative process for its activities under SEP and link SEP to the performance-related goals and measures developed by the agency, track performance towards agency goals, and determine appropriate program changes. The evaluation should include an analysis of inspection findings to identify trends and risks at the national level.

    Agency Affected: Department of Transportation

    Status: Closed - Not Implemented

    Comments: FAA decided to transition all air carriers currently under the Surveillance and Evaluation Program (SEP) to the ATOS program by December 31, 2007. Once all air carriers have been moved to ATOS SEP will be discontinued. As a result, FAA decided not to establish an evaluation process for SEP.

    Recommendation: To improve the effectiveness of the agency's oversight of non-legacy airlines, the Secretary of Transportation should direct the FAA Administrator, to better utilize geographic inspectors' support, to improve the geographic inspectors' understanding of the system safety approach and operations of the airlines they inspect. FAA should consider actions such as additional training, additional oversight in particular areas, having airlines' operating manuals available online for review by inspectors, and improving communication between geographic inspectors and principal inspectors on issues related to identifying safety violations.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: FAA revised the SEP program (revision 22) to require principal inspectors who originate inspection requirements to provide specific instructions to geographic inspectors who will carry out those inspections. The purpose of the instructions is to ensure correct data entry and coding. The revised order also calls for training geographic inspectors on these changes.

    Apr 8, 2014

    Feb 28, 2014

    Feb 12, 2014

    Feb 5, 2014

    Feb 3, 2014

    Jan 31, 2014

    Jan 16, 2014

    Dec 9, 2013

    Looking for more? Browse all our products here