Defense Transportation:

Study Limitations Raise Questions about the Adequacy and Completeness of the Mobility Capabilities Study and Report

GAO-06-938: Published: Sep 20, 2006. Publicly Released: Sep 20, 2006.

Additional Materials:

Contact:

William M. Solis
(202) 512-8365
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Defense (DOD) issued the Mobility Capabilities Study (MCS), which was intended to identify and quantify the mobility capabilities needed to support U.S. strategic objectives into the next decade. The MCS found that projected capabilities are adequate to achieve U.S. objectives with an acceptable level of risk--that is, current U.S. inventory of aircraft, ships, prepositioned assets, and other capabilities are sufficient, in conjunction with host nation support, and assuming planned investments take place. The Senate report accompanying the bill for the fiscal year 2005 Defense Authorization Act required GAO to report on the adequacy and completeness of the MCS. GAO assessed the extent to which the MCS met generally accepted research standards that this type of study would be expected to meet to be considered sound and complete.

DOD used an innovative approach in conducting the study and acknowledged methodological limitations in its report; however, it did not fully disclose how these limitations could affect the MCS conclusions and recommendations. Therefore, it is not transparent how the analyses done for the study support DOD's conclusions. Measured against relevant generally accepted research standards, GAO has identified limitations in the MCS and its report that raise questions about their adequacy and completeness. GAO suggests that Congress and other decision makers exercise caution in using the MCS to make investment decisions. Among GAO's findings: Aspects of modeling and data were inadequate in some areas because data were lacking and the models used could not simulate all relevant aspects of the missions. The report did not explain how these limitations could affect the study results or what the impact on projected mobility capabilities might be. Generally accepted research standards require that models used are adequate for the intended purpose, represent a complete range of conditions, and that data used are properly generated and complete. For example, the MCS modeled hypothetical homeland defense missions rather than homeland defense demands derived from a well defined and approved concept of operations for homeland defense, because the specific details of the missions were still being determined and the data used may be incomplete. The MCS also was unable to model the flexible deterrent options/deployment order process to move units and equipment into theater because of lack of data, but the study assumed a robust use of this process. In addition, the MCS report contains over 80 references to the need for improved modeling or data. While the MCS concluded that combined U.S. and host nation transportation assets were adequate, in describing the use of warfighting metrics in its analyses, the report does not provide a clear understanding of the direct relationship of warfighting objectives to transportation capabilities. Additionally, the report stated that further analysis is required to understand the operational impact of increased or decreased strategic lift on achieving warfighting objectives. Relevant generally accepted research standards require that conclusions be supported by analyses. The use of both warfighting and mobility metrics would allow decision makers to know whether combat tasks were achieved and how much strategic transportation is needed to accomplish those tasks. In some cases, the MCS results were incomplete, unclear, or contingent on further study, making it difficult to identify findings and evaluate evidence. Relevant research standards require results to be presented in a complete, accurate, and relevant manner. For example, the report contains recommendations for further studies and assessments, five of which are under way. However, DOD has no plans to report the impact of these studies on the MCS results after the studies are complete. In addition, the report contains qualified information that is not presented clearly, such as varying assessments of intra-theater assets in three different places.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: DOD recently completed a new mobility capabilities and requirements study. While they developed requirements for homeland defense and used flexible deterrent options in the scenarios, they did not develop models and data for them.

    Recommendation: To provide decision makers with adequate and complete information concerning mobility capabilities so they are able to clearly understand the operational implications of the study and make fully informed programmatic investment decisions, and to improve the usefulness of future mobility capabilities studies, the Secretary of Defense should, when conducting future mobility capabilities studies beginning with any study currently underway, develop models and data for all critical missions, such as homeland defense, and processes, such as the flexible deterrent options/deployment order process.

    Agency Affected: Department of Defense

  2. Status: Closed - Not Implemented

    Comments: In the recently released Mobility Capabilities and Requirements Study-2016, DOD did not identify limitations nor did they explain how any limitations might impact the results. The new study did not refer to previous studies to describe how the new study enhanced any previous study.

    Recommendation: To provide decision makers with adequate and complete information concerning mobility capabilities so they are able to clearly understand the operational implications of the study and make fully informed programmatic investment decisions, and to improve the usefulness of future mobility capabilities studies, the Secretary of Defense should, when conducting future mobility capabilities studies beginning with any study currently underway, include in study reports an explanation of how stated limitations might impact the study results and, at a minimum, describe how recommended future studies might be conducted to enhance the results of the original study.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: In the recently issued Mobility Capabilities and Requirements study, DOD used both warfighting metrics and the mobility metric of million ton miles per day to identify requirements.

    Recommendation: To provide decision makers with adequate and complete information concerning mobility capabilities so they are able to clearly understand the operational implications of the study and make fully informed programmatic investment decisions, and to improve the usefulness of future mobility capabilities studies, the Secretary of Defense should, when conducting future mobility capabilities studies beginning with any study currently underway, incorporate both mobility and warfighting metrics in determining capabilities.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 19, 2014

Sep 18, 2014

Sep 10, 2014

Sep 9, 2014

Sep 8, 2014

Jul 31, 2014

Looking for more? Browse all our products here