Army and Marine Corps Training: Better Performance and Cost Data Needed to More Fully Assess Simulation-Based Efforts
Highlights
What GAO Found
Over the past several decades, the Army and Marine Corps have increased their use of simulation-based training--simulators and computer-based simulations. Historically, the aviation communities in both services have used simulators to train servicemembers in tasks such as takeoffs, and emergency procedures that could not be taught safely live. In contrast, the services' ground communities used limited simulations prior to 2000. However, advances in technology, and emerging conditions in Iraq and Afghanistan have led to increased use of simulation-based training in the ground forces. For example, in response to increases in vehicle rollovers, both services began using simulators to train servicemembers to safely evacuate vehicles. The services are also collaborating in the development of some simulation-based training devices. For instance, according to Marine Corps officials, the service reused 87 percent of the Army's Homestation Instrumentation Training System's components in its own training system, achieving about $11 million in cost avoidance and saving an estimated 7 years in fielding time. The services are also taking steps to better integrate live and simulation-based training, developing technical capabilities to connect previously incompatible simulation-based training devices. The Army's capability is now being fielded, and the Marine Corps' is in the initial development phase.
The Army and Marine Corps consider various factors in determining whether to use live or simulation-based training, but lack key performance and cost information that would enhance their ability to determine the optimal mix of training and prioritize related investments. As the services identify which requirements can be met with either live or simulation-based training or both, they consider factors such as safety and training mission. Also, they have cited numerous benefits of simulation-based training, such as improving servicemember performance in live training events, and reducing operating costs. Both services rely on subject matter experts, who develop their training programs, and after action reports from deployments and training exercises for information on how servicemembers may have benefited from simulation-based training. However, neither service has established outcome metrics to assist them in more precisely measuring the impact of using simulation-based devices to improve performance or proficiency. Leading management practices recognize that performance metrics can help agencies determine the contributions that training makes to improve results. Army and Marine Corps officials also generally consider simulation-based training to be less costly than live training and analyze some data, such as life cycle costs, when considering options to acquire a particular simulation-based training device. However, once simulation-based training devices are fielded, the services neither reevaluate cost information as they determine the mix of training nor have a methodology for determining the costs associated with simulation-based training. Federal internal control standards state that decision makers need visibility over a program's financial data to determine whether the program is meeting the agencies' goals and effectively using resources. Without better performance and cost data, the services lack the information they need to make more fully informed decisions in the future regarding the optimal mix of training and how best to target investments for simulation-based training capabilities.
Why GAO Did This Study
The Army and Marine Corps use live and simulation-based training to meet training goals and objectives. Service officials have noted benefits from the use of simulation-based training--both in terms of training effectiveness and in cost savings or cost avoidance. A House report accompanying the bill for the National Defense Authorization Act for 2012 mandated GAO to review the status of the military services' training programs. This report follows GAO's reports on the Navy and Air Force, and assesses (1) changes in the Army's and Marine Corps' use of simulation-based training, including efforts to integrate live and simulation-based training capabilities; and (2) the factors the Army and Marine Corps consider in determining whether to use live or simulation-based training, including the extent to which they consider performance and cost information. GAO focused on a broad cross-section of occupations (e.g., aviation, armor, artillery), and analyzed service training strategies and other documents; and conducted six site visits and interviewed service officials involved with training and training development for the selected occupations.
Recommendations
GAO recommends that the services develop metrics, and a methodology to compare live and simulation-based training costs. DOD partially concurred, but noted that it captures all relevant costs needed for decision making. GAO continues to believe the services may not be considering some important simulation-based training costs and a specific methodology is needed to more fully identify the universe of costs needed for comparison purposes.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training, the Secretary of Defense should direct the Secretary of the Army and the Commandant of the Marine Corps to develop outcome-oriented performance metrics that can be used to assess the impact of simulation-based training on improving the performance or proficiency of servicemembers and units. |
The Department of Defense (DOD) partially agreed with GAO's July 2013 recommendations. As of June 2021 the Army and Marine Corp has completed some actions and had other ongoing action intended to address the recommendation. For example, according to DOD documentation, the Army has several steps in process to improve performance with simulations based training, to include: (a) using virtual training to conduct collective training exercises thereby increasing proficiency in maneuver tasks without expending live training miles; (b) incorporating simulation capabilities in how Army units prepare for large scale combat operations, including synchronization of command post exercises with 5 separate units at 4 locations; and (c) integrating training capabilities into operational worn heads-up displays to provide a more efficient and on-had training capability for units. As of June 2021 these actions remain in progress. The Army expects to fully complete these initiatives by the fourth quarter of fiscal year 2023. However, the documentation provided by DOD does not include the identification of performance metrics associated with these efforts. Once the Army provides a description of the performance metrics and fully implements the identified efforts to improve simulations based training, they should be able to more effectively assess the impact of simulations based training on improving the performance or proficiency of servicemembers and units. In June 2020, the Marine Corps published the Marine Corps Training Environment Modernization Plan which provided the strategic level vision for future Marine Corps training that integrates live, virtual, and constructive training environments in support of the 38th Commandant's planning guidance. According to DOD officials, this plan has been converted into a Marine Corps Order that is currently in staffing. Documentation provided by DOD officials, states that this order will outline decision criteria for investment and divestment related to current virtual training systems, and develop a method to evaluate the anticipated value of future training systems prior to acquisition decisions. Further, the DOD documentation states that the Marine Corps has updated the format for its Verification, Validation, and Accreditation process, to reflect the prioritization of training metrics-including user feedback-over system metrics. According to the documentation, the Marine Corps Order would also formalize the process of integrating proficiency data that analytically demonstrates if a system is capable of improving training and readiness performance. The Marine Corps expects to complete these actions by July 2021. Once these actions are fully implemented, the Marine Corps should be able to more effectively assess the impact of simulations based training on improving the performance or proficiency of servicemembers and units.
|
| Department of Defense | To improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training, the Secretary of Defense should direct the Secretary of the Army and the Commandant of the Marine Corps to develop a methodology--to include identifying the costs that should be included and how these costs should be captured--for comparing the costs associated with the use of live and simulation-based training. |
The Department of Defense (DOD) partially agreed with GAO's July 2013 recommendations. As of June 2021 the Army and Marine Corp has completed some actions and had other ongoing action intended to address the recommendation. For example, according to DOD documentation, the Army has several steps in process to improve performance with simulations based training, to include: (a) using virtual training to conduct collective training exercises thereby increasing proficiency in maneuver tasks without expending live training miles; (b) incorporating simulation capabilities in how Army units prepare for large scale combat operations, including synchronization of command post exercises with 5 separate units at 4 locations; and (c) integrating training capabilities into operational worn heads-up displays to provide a more efficient and on-had training capability for units. As of June 2021 these actions remain in progress. The Army expects to fully complete these initiatives by the fourth quarter of fiscal year 2023. However, the documentation provided by DOD does not include a methodology for conducting a cost/benefit analysis of these efforts. Once the Army provides a description of the approach to analyzing the costs and benefits associated with the identified efforts to improve simulations based training, they should be able to improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training. In June 2020, the Marine Corps published the Marine Corps Training Environment Modernization Plan which provided the strategic level vision for future Marine Corps training that integrates live, virtual, and constructive training environments in support of the 38th Commandant's planning guidance. According to DOD officials, this plan has been converted into a Marine Corp Order that is currently in staffing. Documentation provided by DOD officials, states that this order will outline decision criteria for investment and divestment related to current virtual training systems, and develop a method to evaluate the anticipated value of future training systems prior to acquisition decisions. According to the DOD documentation, the Marine Corps Order will also formalize a process used by the Marine Corps to collect and analyze usage data and conduct effectiveness analysis of virtual training devices. Specifically, the Marine Corps has made virtual training systems part of the Range Facility Management Support System. Through this system, the Marine Corps can collect training, scheduling, utilization, and reporting data. The DOD documentation states that Marine Corps Training and Education Command plans to use these data to track actual throughput-who reserved a virtual training system and who actually showed up-by system and location, and to conduct a cost/benefit analysis to assess demand versus sustainment cost. As of June 2021 these actions remain in progress. Once these actions are fully implemented, the Marine Corps should be able to improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training.
|