Aviation Safety:

Additional FAA Efforts Could Enhance Safety Risk Management

GAO-12-898: Published: Sep 12, 2012. Publicly Released: Sep 12, 2012.

Additional Materials:

Contact:

Gerald Dillingham, Ph.D.
(202) 512-4803
dillinghamg@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Federal Aviation Administration (FAA) and its business lines and offices are in different stages of their implementation of Safety Management Systems (SMS). FAA finalized its agency-wide implementation plan in April 2012, and the Air Traffic Organization (ATO) has completed its SMS implementation, but other FAA SMS efforts are in the early stages. FAA business lines, such as the Aviation Safety Organization (AVS) and the Office of Airports (ARP), have SMS guidance and plans largely in place and have begun to integrate related practices into their operations, but many implementation tasks remain incomplete, and officials and experts project that full SMS implementation could take many years.

There are a number of key practices that can help agencies plan for and efficiently implement new projects, including large scale transformations such as FAA's SMS implementation, and FAA has many in place. For example, FAA has support from top leadership and a clear project mission. However, FAA has only partially addressed other key practices such as developing a project plan to track SMS implementation, and FAA has not addressed performance-related practices such as establishing SMS performance measures or links between employees' performance standards and SMS.

Several challenges remain that may affect FAA's ability to effectively implement SMS. FAA is taking steps to address some challenges and stakeholder concerns, but challenges related to data sharing and data quality; capacity to conduct SMS-based analyses and oversight; and standardization of policies and procedures could negatively affect FAA's efforts to implement SMS in a timely and efficient manner. Further, FAA officials stated that SMS implementation will require some skills that agency employees do not have, but FAA has not yet assessed the skills of its workforce to identify specific gaps in employee expertise. In addition, while existing federal law protects any data collected for SMS, any data airports collect could be subject to state-specific Freedom of Information Act laws, a gap that could create a disincentive for airports to fully participate in SMS implementation.

Why GAO Did This Study

The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The FAA is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach--referred to as SMS. SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation.

As requested, this report addresses (1) the status of FAA's implementation of SMS, (2) the extent to which FAA's SMS efforts have been consistent with key practices for successful planning and implementation of a new program, and (3) challenges FAA faces in implementing SMS. To address these issues, GAO reviewed FAA SMS documents, compared FAA efforts to key practices, and interviewed agency and industry officials.

What GAO Recommends

GAO recommends that FAA develop systems to: track SMS implementation, evaluate employee performance as it relates to SMS, and assess whether SMS meets its goals and objectives; conduct a workforce analysis for SMS; and consider strategies to address airports' data concerns. The Department of Transportation agreed to consider the recommendations and provided clarifying information about SMS, which GAO incorporated.

For more information, contact Gerald L. Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: Through the activities identified in the FAA SMS and AVS SMS Implementation Plans, FAA expects to have tools and processes in place to evaluate the effectiveness of SMS by April 2015. According to FAA officials, through the FAA Administrator?s Risk-Based Decision Making Initiative, the FAA is focusing its effort on measuring system performance since the objective of SMS is to increase safety (or reduce risk) in the aviation system. This Risk-Based Decision Making Initiative will leverage SMS processes and tools to ensure that the safety impact of decisions is understood by decision-makers. The FAA is developing a method of collecting and prioritizing safety concerns that span FAA Lines of Business. The goal is to create a process to create a list of Cross-Organizational Significant Safety Issues and then apply Safety Risk Management to those issues. In FY2015, the FAA will establish more formal, repeatable processes to identify Significant Safety Issues within Lines of Business. As these activities evolve, the results will be incorporated into the FAA's performance management processes and aligned with existing performance measures. The activities to establish formal repeatable processes for the identification of Significant Safety Issues is expected to be completed in FY2015.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better evaluate the effectiveness of the agency's efforts to implement SMS, the Secretary of Transportation should direct the FAA Administrator to develop a system to assess whether SMS meets its goals and objectives by identifying and collecting related data on performance measures.

    Agency Affected: Department of Transportation

  2. Status: Open

    Comments: According to FAA officials, the FAA will continue to pursue incorporation of SMS into employee performance plans at all levels and ensure alignment with FAA strategic goals. Employees who are responsible for the implementation of SMS will have SMS metrics included in their performance plans by September 30, 2014. In August 2014, FAA officials reported that FAA has incorporated SMS implementation targets/metrics into performance plans of the appropriate management officials through the alignment of strategic, business, and individual plans. The FAA has established SMS/Risk-Based Decision Making Initiative organizational targets in Business Plans. In FY2014, an SMS/Risk-Based Decision Making Initiative Organizational Success Increase measure was established to identify a significant safety issue with accompanying mitigation strategies by September 30, 2014. GAO is awaiting further information on the performance plans and measures.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to align strategic goals with employee efforts, the Secretary of Transportation should direct the FAA Administrator to develop a system to evaluate employees' performance as it relates to SMS.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: In 2012, GAO reported that Federal Aviation Administration's (FAA) implementation of its new Safety Management System (SMS) approach was consistent with many, but not all key practices for successful project implementation. There are a number of key practices and implementation steps that can help agencies successfully plan for and implement new projects, including large scale transformative ones such as FAA's implementation of SMS. As GAO has previously reported, addressing these practices can help an agency improve its efficiency, effectiveness, and accountability. At the time of GAO's audit, FAA had many of these key factors in place, such as established support from top leadership and a clear project mission. However, FAA had only partially addressed other key practices such as developing a project plan to track SMS implementation. FAA officials stated that they had plans to develop a system to monitor and track the progress of activities needed to implement SMS, but FAA did not have a system for tracking agency-wide SMS implementation, a key practice particularly important during the initial planning phase of project implementation. Therefore, GAO recommended that the Secretary of Transportation direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines. In 2015, GAO confirmed that FAA issued a policy (Order 8000.369A) requiring the development of a system for tracking and reporting on SMS implementation within and across business lines. To implement this policy, FAA utilized the FAA SMS Committee, which is comprised of managers from each of the four business lines and one staff office currently implementing SMS. The Committee meets twice a month and reports to the FAA SMS Executive Council. The FAA SMS Committee currently monitors SMS implementation and provides reports to the FAA SMS Executive Council regarding the status of activities identified in the FAA SMS Implementation Plan. Meeting notes are maintained by the Office of Accident Investigation and Prevention, FAA's lead office for SMS. As a result, FAA has implemented another key practice enhancing the effectiveness of the agency's efforts to implement SMS and help to maximize the positive impact of SMS implementation on aviation safety.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better manage implementation, the Secretary of Transportation should direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines.

    Agency Affected: Department of Transportation

  4. Status: Open

    Comments: According to information provided by FAA in August 2014, SMS-related competencies have been developed and incorporated into the appropriate plans and processes. In addition, an activity has been established to develop functional requirements and competencies for a safety data and risk analytics workforce under the Administrator?s Risk-Based Decision Making Strategic Initiative. In FY2014, the activity team identified occupations performing risk and data analytics and developed a competency model to help to create and manage competencies and competency profiles. The activity team is currently conducting job task analyses for key positions. In FY 2015, the activity team will develop and publish new job task analyses for safety positions involved in data and risk analysis, as well as create individual organizational development plans. The outputs of this activity team are expected to feed workforce plans and training and hiring activities in the FAA in support of SMS and Risk-Based Decision Making.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better leverage existing resources and facilitate SMS implementation, the Secretary of Transportation should direct the FAA Administrator to conduct a workforce analysis to inventory existing employee skills and abilities and develop strategies for addressing any SMS-related gaps identified.

    Agency Affected: Department of Transportation

  5. Status: Open

    Comments: According to FAA officials, the issue of public release of SMS data is of concern to airports. In FY 2013, the FAA will start discussions with airports and the airport associations to review the specific SMS data that should be considered for protection. According to information provided by FAA officials in October 2013, discussions have taken place with airports and airport associations to review specific SMS data that should be considered for protection. The FAA Office of Airports (ARP) received comments from industry associations and airport operators on data collection, protection, and sharing. The issue is that existing data protection legislation does not address issue. States rights and Freedom of Information laws regarding airport data sharing/protection could impact the path of any new legislation. Given the complexity of these issues, the FAA continues to evaluate whether a legislative proposal is appropriate. On September 4, 2014, GAO requested further information on whether a legislative proposal has been advanced by FAA officials. When FAA provides a response, GAO will provide updated information.

    Recommendation: To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to maximize the positive impact of SMS implementation on aviation safety, the Secretary of Transportation should direct the FAA Administrator to consider strategies to address airports' concerns that may negatively affect data collection and data sharing, including asking Congress to provide additional protections for SMS data collected by public entities.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Feb 10, 2016

Jan 28, 2016

Jan 14, 2016

Jan 6, 2016

Dec 17, 2015

Dec 16, 2015

Dec 10, 2015

Dec 8, 2015

Dec 3, 2015

Nov 30, 2015

Looking for more? Browse all our products here