DOD Operational Testing: Oversight Has Resulted in Few Significant Disputes and Limited Program Cost and Schedule Increases

What GAO Found

The Director, Operational Test and Evaluation (DOT&E) provided oversight for 454 Department of Defense (DOD) acquisition programs from fiscal years 2010-2014. Military service officials identified 42 programs from that period that they believed had significant disputes with DOT&E over operational testing—that is, disputes that may have led to cost and schedule impacts for programs. Operational testing is intended to evaluate a system's capability in realistic combat conditions before full-rate production or full deployment. Acquisition programs and DOT&E have different objectives and incentives, which can potentially fuel tension between the two over what is needed to accomplish operational testing for programs. According to military service officials, the tension is generally manageable and differences usually are resolved in a reasonable and timely manner, with modest adjustments often required in the course of developing and executing a test approach. However, sometimes differences about operational testing requirements, methods, costs, or results develop into significant disputes and are more difficult to resolve. Acquisition and test officials from the military services identified only a limited number of cases—less than 10 percent of programs receiving DOT&E operational test oversight since fiscal year 2010—that they believed had experienced significant operational testing disputes with DOT&E. Officials noted that although these disputes can require additional time and effort to work through, they generally get resolved.

In an in-depth review of 10 case studies selected from among the 42 programs with significant disputes, GAO identified a variety of factors that contributed to disputes between the acquisition programs and DOT&E, but only a few cases that involved considerable cost or schedule impacts. Key factors involved the adequacy of proposed testing and differences over test requirements, assets, and the reporting of test results. In general, GAO found that DOT&E had valid and substantive concerns about operational test and evaluation for each of the 10 cases reviewed. However, military service officials indicated to GAO that testing advocated by DOT&E was, in some instances, beyond what they believed was necessary and lacked consideration for programs' test resource limitations. Many of the disputes GAO reviewed were, or are expected to be, resolved in DOT&E's favor with limited cost and schedule impacts to the programs. In a few cases, military service officials acknowledged that benefits were achieved from resolving the disputes, such as a reduction in the scope of operational testing and better understanding of system requirements. Resolution of disputes for three programs—DDG-51 Flight III Destroyer, F-35 Joint Strike Fighter, and CVN 78 aircraft carrier—had considerable potential or realized cost or schedule effects and required formal involvement from senior DOD leadership. For the first two programs, hundreds of millions of dollars in additional costs were associated with resolving their disputes. For CVN 78, the dispute—which remains unresolved— involves the Navy's carrier deployment schedule and whether survivability testing will be deferred by several years. For the other seven case study programs that GAO reviewed, the cost and schedule effects tied to dispute resolution were more limited, and in some instances, not related to operational testing requirements.

Why GAO Did This Study

DOD conducts extensive operational testing and evaluation of its military systems prior to full-rate production and fielding. DOT&E plays an integral role in operational test and evaluation by issuing policy and procedures, overseeing operational test planning, and independently evaluating and reporting test results. At times, DOT&E and acquisition programs may disagree about what is needed to adequately demonstrate operational capability, which sometimes may affect programs' cost or schedule.

The Joint Explanatory Statement to Accompany the National Defense Authorization Act for Fiscal Year 2015 directed GAO to review DOT&E's oversight activities. This report examines (1) the extent to which DOD acquisition programs have had significant disputes, if any, with DOT&E over operational testing, and (2) the circumstances and impact of identified disputes. GAO evaluated documentation and interviewed officials from DOT&E, other DOD test organizations, and the acquisition community. GAO also conducted 10 case studies from among 42 programs identified by military service officials as having had significant disputes with DOT&E. GAO analyzed information received from acquisition and testing officials to verify the merits and degree of those disputes. Based on this assessment, GAO selected case studies that were representative of the most significant disputes identified across the military services.