Defense Acquisitions: Assessment of Institute for Defense Analyses' C-130 Avionics Modernization Program Analysis

What GAO Found

The 2013 study conducted by the Institute for Defense Analyses (IDA) is comprised of two key components: (1) its estimate of the costs of the C-130 Avionics Modernization Program (AMP) and two reduced scope alternatives referred to as Options A and B, which offer fewer avionics upgrades than AMP and, unlike AMP, would both require a navigator in the cockpit: and (2) its assessment of the operational effectiveness—that is, benefits—of AMP and each alternative. IDA estimated that that AMP’s costs for research, development, test and evaluation (RDT&E), procurement, and operation and support (O&S) over a 25-year period would total $2.137 billion—significantly higher than the estimated $1.531 billion for Option A and $530 million for Option B.  AMP’s more sophisticated technology enables the removal of a navigator from the cockpit, resulting in estimated O&S savings of $1.207 billion over 25 years. However, estimated procurement costs for Options A and B—$1.740 billion less and $2.580 billion less than AMP, respectively—more than offset AMP’s O&S savings. Regarding benefits, IDA’s study found AMP’s operational effectiveness advantage over Options A and B to be marginal. IDA concluded that the Air Force should not pursue the C-130 AMP program as currently defined.

GAO found that IDA’s cost analysis generally followed best practices for high-quality cost estimates, which are critical to the success of any program. Such an estimate provides the basis for informed investment decision making, realistic budget formulation and program resourcing, meaningful progress measurement, proactive course correction when warranted, and accountability for results. GAO analyzed the cost estimating methodology IDA used in its study compared to best practices and found that IDA’s study substantially met two of the characteristics expected of high-quality cost estimates—that they be comprehensive and credible—and partially meet the remaining two characteristics—being well-documented and accurate. GAO focused its assessment on IDA’s cost estimates and did not assess the operational effectiveness portion of IDA’s study as cost was a defining factor in IDA’s conclusions.

Why GAO Did This Study

The Air Force’s C-130 AMP program, which entered development in 2001, was to standardize and upgrade the cockpit and avionics for various configurations of the C-130 fleet. The C-130, a four-engine turboprop aircraft also known as “Hercules” used primarily for military transport, was originally designed in the 1950s. The AMP upgrades were intended to ensure that the C-130 can satisfy the navigation and safety requirements it needs to operate globally, while at the same time reducing crew size by one. The program would also have replaced many systems for which manufacturers no longer exist—a situation referred to as diminishing manufacturing sources.

In 2007, the program experienced a critical breach of a statutory cost threshold (known as a Nunn-McCurdy breach), and in February 2012, the Department of Defense proposed cancelling the C-130 AMP. The National Defense Authorization Act (NDAA) for Fiscal Year 2013 stated that the Secretary of the Air Force may not cancel the program until 90 days after the Secretary provided the Congressional defense committees a cost benefit analysis conducted by the Institute for Defense Analyses (IDA).  The NDAA required the Secretary to seek an agreement with IDA to conduct an independent study of the costs and benefits of modernizing the legacy C-130H airlift fleet compared to the costs and benefits of a reduced-scope program. IDA began its work in March 2013 and produced the required study in September 2013, followed by a detailed annex in December 2013. The National Defense Authorization Act for Fiscal Year 2014 mandated that GAO review IDA’s analysis. This report describes IDA’s study and provides GAO’s assessment.O analyzed the cost estimating methodology IDA used in its study compared to best practices and found that IDA’s study substantially met two of the characteristics expected of high-quality cost estimates—that they be comprehensive and credible—and partially meet the remaining two characteristics—being well-documented and accurate. GAO focused its assessment on IDA’s cost estimates and did not assess the operational effectiveness portion of IDA’s study as cost was a defining factor in IDA’s conclusions.