Navy Readiness: Actions Needed to Evaluate the Effectiveness of Changes to Surface Warfare Officer Training
Fast Facts
NAVY READINESS: Actions Needed to Evaluate the Effectiveness of Changes to Surface Warfare Officer Training
Following ship collisions in 2017 that resulted in the loss of 17 sailors’ lives and significant damage to Navy ships, the Navy has added classroom and simulator training for the Surface Warfare Officers who drive these ships. The Navy plans to triple ship-driving training hours by 2021.
However, the Navy does not yet have a plan to collect fleet-wide feedback on the quality of the new training or routinely test ship-driving skills. We recommended the Navy gather this feedback and take other actions to better measure training quality.
Navy Officials Observe Virtual Ship-Driving Training Scenarios at Naval Base San Diego
Navy officials observe virtual training scenarios
Highlights
What GAO Found
Since 2017, the Navy has made numerous changes and plans additional changes to enhance Surface Warfare Officer (SWO) ship-driving training. The Navy plans for these changes to result in a threefold increase in the number of initial ship-driving training hours for SWOs by 2021, compared with the number of training hours prior to the 2017 collisions (see fig.). The Navy added classroom and simulator time to existing training courses to improve ship-driving skills and is developing two additional simulator-based ship-driving courses planned for 2021. These plans hinge on the completion of two new simulator-based training facilities, scheduled for completion in June 2021 and in January 2023.
USS Fitzgerald Damaged after 2017 Collision and Navy's Subsequent Threefold Increase in Actual and Planned Ship-Driving Training Hours for Surface Warfare Officers' First 3 Years
The Navy has relied on added skill checks conducted throughout a SWO's career to ensure that each SWO has basic ship-driving skills, but has not put key processes and assessments in place to evaluate comprehensively the effectiveness of its changes to ship-driving training. Senior Navy officials stated that it could take 16 years or more to know if the planned changes to SWO training were effective in increasing Commanding Officer ship-driving proficiency across the fleet and stated that they intend to closely monitor the implementation of changes to the training. However, GAO found that in planning an approach for evaluating the changes, the Navy has not:
(1) identified a method to solicit fleet-wide feedback on the quality of the increased ship-driving training received by SWOs;
(2) planned to routinely conduct ship-driving competency “spot checks” that were instituted after the 2017 collisions despite Navy inspectors having found concerns with more than 80 percent of SWOs' ship-driving skills;
(3) provided standard criteria to ship Commanding Officers for qualifying SWOs to drive ships, contributing to significant variance in ship-driving experience and competency levels across the fleet; nor
(4) developed a specific plan to analyze and use information from logbooks in which SWOs are to document ship-driving and related experience.
Without addressing these challenges, the Navy cannot assess in the near term if the significant investments made to expand and enhance SWO ship-driving training are effective; further adjustments are necessary; and Navy ships are being operated safely at sea.
Why GAO Did This Study
In 2017, the Navy had four mishaps at sea including two collisions that resulted in the loss of 17 sailors' lives and hundreds of millions of dollars in damage to Navy ships. In the wake of those mishaps, the Navy identified deficiencies in SWO ship-driving training and related experience as contributing factors and has undertaken a number of efforts to improve these areas.
Senate Report 115-262, accompanying a bill for the Fiscal Year 2019 National Defense Authorization Act, contained a provision that GAO assess SWO training. This report (1) describes the changes the Navy has made to SWO ship-driving training since the 2017 collisions and (2) assesses the extent to which the Navy has taken actions to evaluate the effectiveness of changes made to SWO ship-driving training. GAO reviewed and analyzed changes made to Navy training and assessment practices and related investments; interviewed cognizant officials; and conducted discussions with SWOs aboard 12 ships.
Recommendations
GAO is making four recommendations to the Navy to routinely evaluate SWO training, including that the Navy collect and evaluate fleet-wide feedback on the quality of training; routinely conduct ship-driving competency assessments; provide standard criteria for qualifying ship drivers; and develop a plan to analyze and use logbook information. The Navy concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of the Navy | We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a method to regularly collect feedback from SWOs across the fleet, such as in a survey, regarding the quality of their classroom, simulator, and at-sea training on Division Officer performance; and evaluates trends in the feedback received for the purpose of improving SWO training. (Recommendation 1) |
The Navy concurred with our recommendation. From October 2022 to February 2023, the Navy fielded three surveys to gather feedback regarding the effectiveness of Surface Warfare Officer training in response to our recommendation. The first survey occurs during a Surface Warfare Officer's Advanced Division Officer Course and asks for feedback regarding how well Basic Division Officer and Officer of the Deck Phase I courses prepared Surface Warfare Officers for their first Division Officer tour. The second survey occurs during a Surface Warfare Officer's Command Assessment and asks for feedback on how well the Department Head course prepared Surface Warfare Officers for their first Department Head tour. The third survey occurs during a Surface Warfare Officer's Prospective Commanding Officer course and asks for feedback on how well the Prospective Executive Officer course prepared Surface Warfare Officers for their Executive Officer tours. These three surveys will be regularly used to collect feedback from Surface Warfare Officers across the fleet to evaluate their classroom, simulator, and at-sea training. The actions taken by the Navy to develop a method to regularly collect feedback from Surface Warfare Officers across the fleet, should provide Navy decision makers valuable information that could help them to assess the effects of training on Surface Warfare Officer performance.
|
| Department of the Navy | We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, routinely conduct regular Officer of the Deck competency assessments using samples of sufficient size and using selection methods to gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned ship-driving training programs. (Recommendation 2) |
The Navy concurred with our recommendation. In April 2020, the Navy completed the second in a series of fleet-wide Officer of the Deck competency assessments. Navy officials noted however, that moving forward, they would perform Officer of the Deck assessments as part of their planned 10 mariner skills assessments, evaluations, and competency checks across a Surface Warfare Officers career rather than these ad-hoc fleet-wide competency assessments. In March 2022, the Navy reported that all 10 of its planned mariner skills assessments, evaluations, and competency checks across a Surface Warfare Officer's career were now in place. According to Navy officials the 10 Surface Warfare Officer mariner skills assessments, evaluations, and competency checks result in the Navy assessing more than 1,500 Surface Warfare Officers on their Officer of the Deck performance each year. Navy officials stated that these assessments are collecting at least 10 times the amount of Officer of the Deck assessment data each year-just from the 4 simulator assessments that a Surface Warfare Officer must pass to proceed to the next career milestone-than what the fleet-wide Officer of the Deck competency checks would have yielded. In addition, based on discussions with GAO personnel, Navy officials tailored the assessments to occur prior to any kind of training taking place, so as to see how officers perform based solely on their training and at sea experiences. The actions taken by the Navy to expand, routinize, and tailor the Officer of the Deck competency assessments will help the Navy gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned ship-driving training programs, determine the effectiveness of the changes made to training, and know whether additional changes are needed.
|
| Department of the Navy | We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, provide Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporates these criteria into surface fleet guidance. (Recommendation 3) |
The Navy concurred with our recommendation. In August 2021, Navy officials reported that Commander Naval Surface Forces provided Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification in an update to the Surface Warfare Officer Career Manual in December 2019. In addition, in August 2021, the Navy reported that Commander Naval Surface Forces sent two messages-Mariner Skills Best Practices (Oct. 2020) and Immediate Superiors in Command and Commanding Officers Mariner Skills Evaluation Guidance (Feb. 2021)-to commanding officers that provide Commanding Officers with standard criteria to inform their evaluations of candidates for Officer of the Deck qualification. Finally, In January 2021, Surface Warfare Officer School Command published an instruction entitled Surface Warfare Officer Milestone Mariner Skills Assessments and Competency Checks that provides specific grading criteria and formalizes and standardizes the process for mariner skills assessments. The actions taken by the Navy to provide commanding officers with standard Officer of the Deck assessment criteria and incorporate these criteria into surface fleet guidance reduce the uncertainty in Officer of the Deck qualification expectations and may result in less variation in ship-driving proficiency among Surface Warfare Officers and safer operations at sea.
|
| Department of the Navy | We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a plan to analyze and use Mariner Skills Logbook information to inform decision-making. (Recommendation 4) |
The Navy concurred with our recommendation. In February 2024, the Navy provided documentation that it developed the Ship-handling and Experience Tracker Analyzing Navigation Trends (SEXTANT) program to systematically collect and analyze Mariner Skills Logbook information to provide recommendations to Commander, Naval Surface Forces to assist in decision-making. Through the SEXTANT program, the Navy has collected and analyzed logbook data-for over 1,400 officers to date-and compared the experiential data gained with performance data from the career-long Surface Warfare Officer mariner skills assessments. Metrics from the SEXTANT program have been used to inform decision making in areas such as: determining the number of hours of bridge watch standing required for officers to qualify as Officers of the Deck, evaluating Surface Warfare Officer performance, supporting training efforts, and strengthening detailing and placement decisions. The actions taken to develop a plan to analyze and use information from the Mariner Skills Logbook to inform decision-making better position the Navy to determine the relationship between Surface Warfare Officer experience and ship-driving proficiency.
|