Space Acquisitions:

DOD Needs to Take More Action to Address Unrealistic Initial Cost Estimates of Space Systems

GAO-07-96: Published: Nov 17, 2006. Publicly Released: Nov 17, 2006.

Additional Materials:

Contact:

Cristina T. Chaplain
(202) 512-4859
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Estimated costs for the Department of Defense's (DOD) major space acquisition programs have increased by about $12.2 billion from initial estimates for fiscal years 2006 through 2011. Cost growth for ongoing Air Force programs above initial estimates accounts for a substantial portion of this 44 percent increase. In light of the role that optimistic estimating is believed to have played in exacerbating space acquisition cost growth, you requested that we examine (1) in what areas space system acquisition cost estimates have been unrealistic and (2) what incentives and pressures have contributed to the quality and usefulness of cost estimates for space system acquisitions.

Costs for DOD space acquisitions over the past several decades have consistently been underestimated--sometimes by billions of dollars. For example, Space Based Infrared System High program costs were originally estimated at $4 billion, but the program is now estimated to cost over $10 billion. Estimated costs for the National Polar-orbiting Operational Satellite System program have grown from almost $6 billion at program start to over $11 billion. For the most part, cost growth has not been caused by poor cost estimating, but rather the tendency to start programs before knowing whether requirements can be achieved within available resources--largely because of pressures to secure funding. At the same time, however, unrealistic program office cost estimates have exacerbated space acquisition problems. Specifically, with budgets originally set at unrealistic amounts, DOD has had to resort to continually shifting funds to and from programs, and such shifts have had costly, reverberating effects. Our analyses of six ongoing space programs found that original cost estimates were particularly unrealistic about the promise of savings from increased contractor program management responsibilities, the constancy and availability of the industrial base, savings that could be accrued from heritage systems, the amount of weight growth that would occur during a program, the availability of mature technology, the stability of funding, the stability of requirements, and the achievability of planned schedules. At times, estimates that were more realistic in these areas were available to the Air Force, but they were not used. Cost-estimating and program officials we spoke with identified a number of factors that have contributed to this condition, in addition to larger pressures to produce low estimates that are more likely to win support for funding. Although the National Security Space Acquisition policy requires that independent cost estimates be prepared by bodies outside the acquisition chain of command, it does not require that they be relied upon to develop program budgets. While the policy requires that cost estimates be updated at major acquisition milestones, significant events, such as changes in the industrial base or funding, have occurred between milestones. Within space system acquisitions, cost-estimating officials believe that their roles and responsibilities are not clear and the cost-estimating function is fragmented. Cost-estimating resources have atrophied over the years because of previous downsizing of the workforce, making resources such as staff and data inadequate and the Air Force more dependent on support contractors for the estimating function.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: DOD partially concurred with this recommendation, and stated in its comments that at both the development and the production Milestones for all ACAT I programs U.S.C. Title 10 - Armed Forces requires the Milestone Decision Authority (MDA) [USD(AT&L)] to be informed by an independently developed life cycle cost estimate before making a decision on how to proceed. DOD also stated that the OSD CAIG is charged with developing this estimate and, in practice, present its findings to the Defense Acquisition Board (DAB) when they meet to advise the MDA. According to DOD, the OSD CAIG formally documents its Independent Cost Estimate (ICE) in a report to the MDA. DOD also stated that the complex decision to determine the cost figure used as a basis for funding and to evaluate future program performance must weigh many competing factors that are often qualitative in nature. DOD concluded that as with all other acquisition related decisions, the decision is the MDA's alone, and although thoroughly discussed with the MDA advisers during the DAB meeting and clearly documented in the Acquisition Decision Memorandum, documenting the explicit justification will reduce the MDA's future decision-making flexibility.

    Recommendation: To increase accountability and transparency of decisions in space programs where an independent estimate produced by the Cost Analysis Improvement Group (CAIG) or the Air Force Cost Analysis Agency (AFCAA) is not chosen, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to require officials involved in milestone decisions to document and justify the reasons for their choice and the differences between the program cost estimate and the independent cost estimate.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: The Office of the Secretary of Defense Cost Analysis Improvement Group (OSD CAIG) conducts peer reviews of program office cost estimates with the national security space cost analysis community. This has been ongoing effort inside the OSD CAIG, the result of which is the OSD CAIG independent cost estimate (ICE) memorandum signed by the Chairman of the OSD CAIG to the Under Secretary of Defense (Acquisition, Technology and Logistics)(USD(AT&L)). DOD Instruction 5000.4-M contains the documented process on preparation of ICE memorandums. Additional guidance on cost positions for milestone reviews has been recently issued by the Chairman, OSD CAIG and Director, OUSD (AT&L)/Acquisition Resources and Analysis. The CAIG also collects cost data, facilitated by the Defense Cost and Resource Center (DCARC), which collects current and historical cost data and makes those data available to government analysts. All earned value management system reporting is also now being collected by DCARC. In addition, AT&L in a memorandum dated 1/9/09, directed all programs to submit their cost data in a standardized work breakdown structure. The OSD CAIG has also given $300,000 to the National Reconnaissance Office Cost Group to set a standard for their work breakdown structure and data collection to implement the same tracking, searching and accessing of documents that DCARC currently provides. Finally, in an Air Force policy directive issued 8/5/08, program cost estimates and non-advocate cost assessments for Acquisition Category (ACAT) I programs and selected ACAT II and III programs are to be updated annually and used to support milestone decisions, budget formulation, source selections, program and unit cost breaches, and analysis of alternatives on program expected to meet ACAT I thresholds.

    Recommendation: To better ensure investment decisions for space programs are knowledge-based, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to instill processes and tools necessary to ensure lessons learned are incorporated into future estimates. This could include conducting postmortem reviews of past space program cost estimates (program office and independent cost estimates) to measure cost-estimating effectiveness and to track and record cost-estimating mistakes; developing a centralized cost-estimating database that provides realistic and credible data to cost estimators; establishing protocols by which cost estimators working with the National Reconnaissance Office can share data with the DOD space cost-estimating community while still maintaining appropriate security over classified data; and ensuring estimates are updated as major events occur within a program that could have a material impact on cost, such as budget reductions, integration problems, hardware/software quality problems, and so forth.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: On August 8, 2007, the Under Secretary of the Air Force issued a cost estimating guidance memorandum. The memorandum clearly defines the roles and responsibilities of the Air Force Deputy Assistant Secretary for Cost and Economics (which includes the Air Force Cost Analysis Agency) in providing non-advocate cost assessments in support of acquisition and budget decisions. In addition, the memorandum directs the Deputy Assistant Secretary for Cost and Economics to develop and administer cost analysis policy and guidance, standards and templates, training and education, and to maintain an historical cost database. We believe that all of these actions will increase collaboration within the cost-estimating community and allow the Air Force to benefit from the cost estimators' knowledge and expertise.

    Recommendation: To optimize analysis and collaboration within the space cost-estimating community, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to clearly articulate the roles and responsibilities of the various Air Force cost-estimating organizations, and ensure that space system cost estimators are organized so that the Air Force can gain the most from their knowledge and expertise. In taking these actions for programs for which no independent estimate is developed by the CAIG, consider assigning AFCAA the responsibility for the development of independent cost estimates for space system acquisitions, since it is outside of the acquisition chain of command and therefore likely to be unbiased and not pressured to produce optimistic estimates.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Dec 19, 2014

Dec 16, 2014

Dec 12, 2014

Dec 11, 2014

Dec 1, 2014

Nov 21, 2014

Nov 20, 2014

Nov 19, 2014

Nov 18, 2014

Looking for more? Browse all our products here