Aviation Safety:

Additional FAA Efforts Could Enhance Safety Risk Management

GAO-12-898: Published: Sep 12, 2012. Publicly Released: Sep 12, 2012.

Additional Materials:

Contact:

Gerald Dillingham, Ph.D.
(202) 512-4803
dillinghamg@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Federal Aviation Administration (FAA) and its business lines and offices are in different stages of their implementation of Safety Management Systems (SMS). FAA finalized its agency-wide implementation plan in April 2012, and the Air Traffic Organization (ATO) has completed its SMS implementation, but other FAA SMS efforts are in the early stages. FAA business lines, such as the Aviation Safety Organization (AVS) and the Office of Airports (ARP), have SMS guidance and plans largely in place and have begun to integrate related practices into their operations, but many implementation tasks remain incomplete, and officials and experts project that full SMS implementation could take many years.

There are a number of key practices that can help agencies plan for and efficiently implement new projects, including large scale transformations such as FAA's SMS implementation, and FAA has many in place. For example, FAA has support from top leadership and a clear project mission. However, FAA has only partially addressed other key practices such as developing a project plan to track SMS implementation, and FAA has not addressed performance-related practices such as establishing SMS performance measures or links between employees' performance standards and SMS.

Several challenges remain that may affect FAA's ability to effectively implement SMS. FAA is taking steps to address some challenges and stakeholder concerns, but challenges related to data sharing and data quality; capacity to conduct SMS-based analyses and oversight; and standardization of policies and procedures could negatively affect FAA's efforts to implement SMS in a timely and efficient manner. Further, FAA officials stated that SMS implementation will require some skills that agency employees do not have, but FAA has not yet assessed the skills of its workforce to identify specific gaps in employee expertise. In addition, while existing federal law protects any data collected for SMS, any data airports collect could be subject to state-specific Freedom of Information Act laws, a gap that could create a disincentive for airports to fully participate in SMS implementation.

Why GAO Did This Study

The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The FAA is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach--referred to as SMS. SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation.

As requested, this report addresses (1) the status of FAA's implementation of SMS, (2) the extent to which FAA's SMS efforts have been consistent with key practices for successful planning and implementation of a new program, and (3) challenges FAA faces in implementing SMS. To address these issues, GAO reviewed FAA SMS documents, compared FAA efforts to key practices, and interviewed agency and industry officials.

What GAO Recommends

GAO recommends that FAA develop systems to: track SMS implementation, evaluate employee performance as it relates to SMS, and assess whether SMS meets its goals and objectives; conduct a workforce analysis for SMS; and consider strategies to address airports' data concerns. The Department of Transportation agreed to consider the recommendations and provided clarifying information about SMS, which GAO incorporated.

For more information, contact Gerald L. Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: Through the activities identified in the FAA SMS and AVS SMS Implementation Plans, FAA expects to have tools and processes in place to evaluate the effectiveness of SMS by April 2015. According to FAA officials, through the FAA Administrator?s Risk-Based Decision Making Initiative, the FAA is focusing its effort on measuring system performance since the objective of SMS is to increase safety (or reduce risk) in the aviation system. This Risk-Based Decision Making Initiative will leverage SMS processes and tools to ensure that the safety impact of decisions is understood by decision-makers. The FAA is developing a method of collecting and prioritizing safety concerns that span FAA Lines of Business. The goal is to create a process to create a list of Cross-Organizational Significant Safety Issues and then apply Safety Risk Management to those issues. In FY2015, the FAA will establish more formal, repeatable processes to identify Significant Safety Issues within Lines of Business. As these activities evolve, the results will be incorporated into the FAA's performance management processes and aligned with existing performance measures. The activities to establish formal repeatable processes for the identification of Significant Safety Issues is expected to be completed in FY2015.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better evaluate the effectiveness of the agency's efforts to implement SMS, the Secretary of Transportation should direct the FAA Administrator to develop a system to assess whether SMS meets its goals and objectives by identifying and collecting related data on performance measures.

    Agency Affected: Department of Transportation

  2. Status: Open

    Comments: According to FAA officials, the FAA will continue to pursue incorporation of SMS into employee performance plans at all levels and ensure alignment with FAA strategic goals. Employees who are responsible for the implementation of SMS will have SMS metrics included in their performance plans by September 30, 2014. In August 2014, FAA officials reported that FAA has incorporated SMS implementation targets/metrics into performance plans of the appropriate management officials through the alignment of strategic, business, and individual plans. The FAA has established SMS/Risk-Based Decision Making Initiative organizational targets in Business Plans. In FY2014, an SMS/Risk-Based Decision Making Initiative Organizational Success Increase measure was established to identify a significant safety issue with accompanying mitigation strategies by September 30, 2014. GAO is awaiting further information on the performance plans and measures.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to align strategic goals with employee efforts, the Secretary of Transportation should direct the FAA Administrator to develop a system to evaluate employees' performance as it relates to SMS.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: In 2012, GAO reported that Federal Aviation Administration's (FAA) implementation of its new Safety Management System (SMS) approach was consistent with many, but not all key practices for successful project implementation. There are a number of key practices and implementation steps that can help agencies successfully plan for and implement new projects, including large scale transformative ones such as FAA's implementation of SMS. As GAO has previously reported, addressing these practices can help an agency improve its efficiency, effectiveness, and accountability. At the time of GAO's audit, FAA had many of these key factors in place, such as established support from top leadership and a clear project mission. However, FAA had only partially addressed other key practices such as developing a project plan to track SMS implementation. FAA officials stated that they had plans to develop a system to monitor and track the progress of activities needed to implement SMS, but FAA did not have a system for tracking agency-wide SMS implementation, a key practice particularly important during the initial planning phase of project implementation. Therefore, GAO recommended that the Secretary of Transportation direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines. In 2015, GAO confirmed that FAA issued a policy (Order 8000.369A) requiring the development of a system for tracking and reporting on SMS implementation within and across business lines. To implement this policy, FAA utilized the FAA SMS Committee, which is comprised of managers from each of the four business lines and one staff office currently implementing SMS. The Committee meets twice a month and reports to the FAA SMS Executive Council. The FAA SMS Committee currently monitors SMS implementation and provides reports to the FAA SMS Executive Council regarding the status of activities identified in the FAA SMS Implementation Plan. Meeting notes are maintained by the Office of Accident Investigation and Prevention, FAA's lead office for SMS. As a result, FAA has implemented another key practice enhancing the effectiveness of the agency's efforts to implement SMS and help to maximize the positive impact of SMS implementation on aviation safety.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better manage implementation, the Secretary of Transportation should direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines.

    Agency Affected: Department of Transportation

  4. Status: Closed - Implemented

    Comments: The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The Federal Aviation Administration (FAA) is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach--referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that SMS implementation across FAA requires some skills that agency employees do not have, yet FAA had not formally assessed the skills of its workforce to identify any gaps in the expertise required to implement SMS or determine how to fill those gaps. Stakeholders and experts also questioned whether FAA currently has the resources and capacity needed to fully implement SMS. For example, experts noted that FAA may not have the requisite engineers and other staff to participate in safety risk management efforts, or FAA inspectors to oversee individual airport and air carrier SMS programs. Despite these concerns, FAA has not yet conducted a strategic workforce assessment to accurately determine the skills and staffing levels it needs to manage SMS. Although FAA's SMS implementation plan recommends that business lines create such staffing analyses, none have done so. Nor has FAA conducted an agency-wide workforce assessment for SMS. GAO's internal control standards state that agencies should ensure that skill needs are continually assessed to ensure workforces have the skills necessary to help the agency meet its goals. Therefore, GAO recommended that the Administrator conduct a workforce analysis to inventory existing employee skills and abilities and develop strategies for addressing any SMS-related gaps identified. In 2016, GAO confirmed that FAA developed and administered a competency survey to assess the skills of employees performing aviation safety risk and data analysis to baseline the current competency levels of the workforce. In the Project's Closeout Report, FAA identified gaps in the skills needed to implement SMS and made recommendations to address these gaps, such as creating a job task analysis for their safety analytics workforce, covering both existing analysts and future hires. As a result, FAA now knows whether its staff has sufficient skills or competencies to implement SMS and is taking steps to address the identified gaps.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better leverage existing resources and facilitate SMS implementation, the Secretary of Transportation should direct the FAA Administrator to conduct a workforce analysis to inventory existing employee skills and abilities and develop strategies for addressing any SMS-related gaps identified.

    Agency Affected: Department of Transportation

  5. Status: Closed - Implemented

    Comments: The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The Federal Aviation Administration (FAA) is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach?referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that FAA's implementation of its new SMS approach was consistent with many, but not all key practices for successful project implementation. Several challenges remain that may affect FAA's ability to effectively implement SMS, including challenges related to data sharing. Airport and air carrier representatives are concerned that the public disclosure of safety data could result in negative publicity or legal liability in the event of an incident or accident. While existing federal law protects any data collected for SMS, any data airports collect and any data air carriers share with airports could be subject to state-specific Freedom of Information Act (FOIA) laws, a gap that could create a disincentive for airports to fully participate in SMS implementation. FAA officials said that data protection and legal liability are two of the major concerns throughout the aviation industry that could hinder the implementation of SMS, but airports could find ways to limit the public release of air carrier safety information. In FAA's official response to comments on two Notices of Proposed Rulemaking, FAA stated that airport officials are best situated to understand how to comply with state laws. Nonetheless, GAO found consensus among NTSB and many aviation stakeholders that FAA should seek congressional action regarding the protection of airport data from state FOIA laws. To enhance the effectiveness of efforts to implement SMS, GAO recommended that the Secretary of Transportation direct the FAA Administrator to consider strategies to address airports' concerns that may negatively affect data collection and data sharing, including asking Congress to provide additional protections for SMS data collected by public entities. In 2015, GAO confirmed that FAA held discussions with representatives from airports and airport associations to review specific SMS data that should be considered for protection. The FAA Office of Airports received comments from these stakeholders on data collection, protection, and sharing. After consideration and study of the issue, FAA concluded that it will not request a legislative proposal for additional protections for SMS data. By considering this information, FAA is in a better position to understand stakeholder concerns regarding SMS data collection, protection and sharing.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to maximize the positive impact of SMS implementation on aviation safety, the Secretary of Transportation should direct the FAA Administrator to consider strategies to address airports' concerns that may negatively affect data collection and data sharing, including asking Congress to provide additional protections for SMS data collected by public entities.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Apr 28, 2016

Apr 25, 2016

Mar 16, 2016

Feb 17, 2016

Feb 10, 2016

Jan 28, 2016

Jan 14, 2016

Jan 6, 2016

Dec 17, 2015

Looking for more? Browse all our products here