Military Aviation Mishaps: DOD Needs to Improve Its Approach for Collecting and Analyzing Data to Manage Risks
Highlights
What GAO Found
GAO found the following gaps in the Department of Defense's (DOD) approach for collecting, reporting, and analyzing aviation mishap data:
- The military services' safety centers do not collect standardized aviation mishap data. GAO found that the safety centers did not collect standardized data for 10 to 17 of the 35 agreed-upon data elements for aviation mishaps that were to be provided to the Office of the Secretary of Defense (OSD), depending on the service. As a result, OSD must perform time-consuming manipulation and interpretation of certain data elements received from the safety centers to facilitate comparative analyses, which introduces the risk of errors in the analysis and affects the timeliness of providing critical information to decision makers.
Status of Aviation Mishap Data Element Collection by the Military Services’ Safety Centers, as of May 2018
Note: These numbers represent the 35 data elements that the Office of the Secretary of Defense has identified as related to aviation mishaps. The Naval Safety Center also manages the Marine Corps’ portion of the Naval Aviation Safety Program. For the purposes of GAO’s analysis, standardized data refers to data elements that did not require some level of interpretation or mapping by the Office of the Secretary of Defense to conform to the agreed upon list of values for each data element.
- Lack of consensus on reporting causal factors to OSD for DOD-wide analysis. According to military service and OSD officials, there is no consensus between the safety centers and OSD on OSD's role for conducting causal analysis of aviation mishaps, which has contributed to limitations in the analysis of aviation mishaps that OSD performs. This lack of consensus has led the safety centers to not report all of the agreed-upon data elements to OSD, including the causal factors related to aviation mishaps. Specifically, none of the safety centers were reporting information to OSD on human factors, such as performance-based errors, that contributed to the mishaps. OSD officials noted that without receiving the underlying human factor categories, they have been limited in the type of analyses that they can develop. For example, OSD conducts descriptive analyses on overall trends in mishaps, but it has been unable to conduct analyses on patterns in human factors--including patterns that may cross cut the military services--due to the data not being provided by the safety centers. Until there is consensus on OSD's role, it may be unable to obtain and analyze DOD-wide mishap data.
- DOD does not consistently collect relevant training data to analyze trends in mishaps. GAO found that certain training data related to pilots' training records are not being collected in all mishap investigations, such as information on a pilots' recent flying experience or training proficiency in the task or mission performed during a mishap. GAO also found variations in how certain training-related data were being recorded in the safety centers' data systems. Recent studies have suggested that training shortfalls are a potential indicator of trends in aviation mishaps, but additional training data would be required for further analysis.
Why GAO Did This Study
According to DOD, an aviation mishap is an unplanned event or series of events that result in damage to DOD property, illness or injury to DOD military or civilian personnel, damage to public property, or injury or illness to non-DOD personnel caused by DOD activities. Recent occurrences of military aviation mishaps have raised concerns about their underlying causes, such as the relationship between completed training and flight safety. DOD is considering options that would reorganize certain policy and oversight functions related to aviation mishaps.
The House committee report accompanying its version of the National Defense Authorization Act for Fiscal Year 2018 included a provision for GAO to assess what is known about the relationship between training completed on rotary-wing aircraft and the number of mishaps that have occurred. This report discusses the extent to which DOD has a comprehensive approach for the collection, reporting, and analysis of mishap data to inform aviation risk-management decisions. GAO analyzed DOD and service instructions and memorandums, reviewed mishap data collected by the safety centers and reported to OSD as of May 2018, and interviewed officials from OSD and the safety centers to understand the types of data collected during mishap investigations.
Recommendations
GAO recommends that DOD (1) take interim steps to help ensure that standardized aviation mishap data elements are collected by the safety centers; (2) update department-wide and service instructions and policies to clarify the responsibility of OSD for conducting analyses and its access to the military services' causal information on human factors that contributed to aviation mishaps; and (3) identify relevant training-related data to collect as part of any update of the aviation mishap data elements and incorporate these data into future analyses. DOD concurred with the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense |
Priority Rec.
The Secretary of Defense ensures that the Offices of the Under Secretary of Defense for Personnel and Readiness and Under Secretary of Defense for Acquisition and Sustainment in coordination with the Secretaries of the Army, Navy, and Air Force take interim steps to help ensure that standardized aviation mishap data elements are collected by the safety centers. (Recommendation 1) |
DOD concurred with this recommendation. According to DOD documentation provided in February 2024, the department has taken some steps to implement this recommendation. Specifically, the Defense Safety Oversight Council DOD Safety Information Management Working Group has completed a safety management business process re-engineering effort that includes standardizing aviation mishap data elements in all of the department's safety information management systems. Further this working group has standardized the procedures for collecting this data. DOD's safety information management systems are scheduled to implement the safety data standards by July 2024. Completing these actions would allow the military services' safety centers to collect relevant training-related data as part of the update of aviation mishap data elements, as GAO recommended in August 2018.
|
| Department of Defense | The Secretary of Defense ensures that the Offices of the Under Secretary of Defense for Personnel and Readiness and Under Secretary of Defense for Acquisition and Sustainment in coordination with the Secretaries of the Army, Navy, and Air Force update department-wide and service instructions and policies to clarify the responsibility of the Office of the Secretary of Defense for conducting analysis and its access to the military services' information on human factors that contributed to aviation mishaps. (Recommendation 2) |
DOD concurred with this recommendation. According to DOD documentation provided in February 2024, the department has taken some steps to implement this recommendation. According to the plan, DOD is revising DOD Instruction 6055.07, which will clarify the department's policy that it is the responsibility of the Office of the Secretary of Defense to access and conduct analysis on the military services' information on human factors that contributed to aviation mishaps. DOD's estimated completion date for this effort is March 2025. Completing this action would clarify the Office of the Secretary of Defense's role for conducting analyses and access to the military services' safety centers' data on aviation mishaps, as GAO recommended in August 2018.
|
| Department of Defense | The Secretary of Defense ensures that the DOD Chief Management Officer in coordination with the offices of the Under Secretary of Defense for Personnel and Readiness and Under Secretary of Defense for Acquisition and Sustainment and the Secretaries of the Army, Navy, and Air Force identify relevant training-related data to collect as part of any update of the aviation mishap data elements and incorporate these data into future analyses. (Recommendation 3) |
DOD concurred with this recommendation. According to DOD documentation provided in February 2024, the department has taken some steps to implement this recommendation. Specifically, the Office of the Under Secretary of Defense for Personnel and Readiness proposed updated safety data standards to the military departments, which approved of the new standards. DOD is in the process of coordinating with the military departments to implement the new standards into their information systems. DOD's safety information management systems are scheduled to implement the safety data standards by July 2024. Completing these actions would allow the military services' safety centers to collect relevant training-related data as part of the update of aviation mishap data elements, as GAO recommended in August 2018.
|