Missile Defense Program Instability Affects Reliability of Earned Value Management Data
GAO-10-676, Jul 14, 2010
By law, GAO is directed to assess the annual progress the Missile Defense Agency (MDA) made in developing and fielding the Ballistic Missile Defense System (BMDS). GAO issued its latest assessment of MDA's progress covering fiscal year 2009 in February 2010. This report supplements that assessment to provide further insight into MDA's prime contractor performance for fiscal year 2009. Prime contractors track earned value management (EVM) by making comparisons that inform the program as to whether the contractor is completing work at the cost budgeted and whether the work scheduled is being completed on time. Our analysis of contractor EVM data included examining contract performance reports for 14 BMDS contracts, reviewing the latest integrated baseline reviews, performing extensive analysis of data anomalies, and conducting interviews with Defense Contract Management Agency (DCMA) officials--the independent reviewers of MDA contractor EVM systems.
Unlike GAO's reports in previous years, GAO was unable to analyze the EVM data for all MDA contracts. GAO determined that the data for the Ground-based Midcourse Defense (GMD) and Targets and Countermeasures programs were not sufficiently reliable to include in our report because of instability in these programs' baselines. When the baseline on which the work is performed and measured against is no longer representative of the program of record, program managers and other decision makers lose the ability to develop constructive corrective action plans to get the program on track. Specifically, without reliable EVM data, GAO was unable to identify significant performance drivers or forecast future cost and schedule performance. Because the two contracts associated with these programs represent half of the budgeted cost at completion for the 14 contracts GAO reviewed, GAO also determined it was not appropriate in this report to aggregate total projected underruns or overruns of the remaining 12 prime contracts as GAO has in prior reports. The GMD prime contractor performance data was not sufficiently reliable to use as the basis for analysis because the contractor was unable to update its baseline to include numerous changes to the program and modifications to the contract. Despite three large restructures since 2007 totaling over $2 billion, the GMD program has not conducted an integrated baseline review since December 2006. DOD acquisition policy states that an integrated baseline review is to be conducted within 6 months after contract award, exercise of contract options, or a major modification to an existing contract. The Director, MDA has taken extra steps to gain insight into the contractor's performance. Further, he intends to report EVM information to Congress annually. Similarly, the EVM data for the Targets and Countermeasures contractor is also not sufficiently reliable to use in our analysis. DCMA identified several issues with the stability of the Targets and Countermeasures program baseline including a large amount of schedule and quantity changes to planned flight tests and over 20 contract changes to the scope of work or corrective actions to quality issues for one of the delivery orders over the course of a year. Because the contractor has not been able to update the established budget in the baseline, the cost performance reports do not reflect an appropriate baseline against which to measure cost and schedule progress. Nine of the remaining twelve contracts experienced cost overruns for fiscal year 2009 mostly because of issues with maturing technologies, immature designs, or other technical issues. For example, the Airborne Laser contractor experienced a failure in some of the system's optics which required it to develop and procure new high power optics, delaying the test schedule and increasing program cost. GAO recommends that MDA resolve prime contractor EVM data reliability issues by the beginning of fiscal year 2011. If, by this time, MDA has not resolved these issues, the Secretary of Defense should provide a report to Congress on the steps MDA is taking to resolve them. DOD concurred with our recommendation.
- Closed - implemented
- Closed - not implemented
Recommendation for Executive Action
Recommendation: The Secretary of Defense should direct MDA to resolve prime contractor data reliability issues by the beginning of fiscal year 2011 and, if MDA has not resolved the data reliability problems, determine the barriers preventing resolution and provide a report to Congress on: (1) the steps MDA is taking to make its contractor data sufficiently reliable, (2) how the data reliability issues affect MDA's ability to provide oversight of its contractors, and (3) the effect these issues have on MDA's ability to report contractor progress to others, including Congress.
Agency Affected: Department of Defense
Status: Closed - Not Implemented
Comments: In providing comments on this report, the Department of Defense concurred with this recommendation and has closed this recommendation based on actions taken by MDA. However, our assessment is that corrective actions were not completed prior to the beginning of fiscal year 2011, as stipulated in our recommendation. While DOD stated that the Missile Defense Agency (MDA) considered its fiscal year 2009 prime contractor performance data to be reliable, the agency still chose to implement an agency-wide integrated baseline review (IBR) process through fiscal year 2010 and into fiscal 2011. Over a course of several months prior to fiscal year 2011, seven IBRs were conducted on prime and major subcontractors for the Ground-Based Midcourse Defense program element with three more scheduled for fiscal year 2011 - which is after the completion date set in the recommendation. Although MDA was confident that these IBRs, conducted with the Defense Contract Management Agency (DCMA), demonstrated the fidelity of the data, the failure of GMD's flight test (FTG-06a) in December 2010 ultimately led to a restructure of the program which would again necessitate new IBRs before the earned value management (EVM) data can be collected and the reliability assessed. The Targets and Countermeasures (TC) program element also conducted five IBRs through fiscal year 2010 and fiscal year 2011, with the onsite support of DCMA. Three more TC IBRs were planned for fiscal year 2011 which also exceeded the timeline set in our recommendation. DOD currently lists this recommendation as closed, citing the events listed above, MDA EVM review processes that the agency believes ensure contract data reliability, and MDA leadership reviews of contractor progress using EVM data in quarterly Baseline Execution Reviews.