Joint Intelligence Analysis Complex: DOD Needs to Fully Incorporate Best Practices into Future Cost Estimates
Highlights
What GAO Found
GAO assessed the cost estimate for the military construction project to consolidate and relocate the Joint Intelligence Analysis Complex (JIAC) at Royal Air Force (RAF) base Croughton and found that it partially met three and minimally met one of the four characteristics of a reliable cost estimate defined by GAO best practices, as shown in the table below. For example, it minimally met the credibility standard because it did not contain a sensitivity analysis; such analyses reveal how the cost estimate is affected by a change in a single assumption, without which the estimator will not fully understand which variable most affects the estimate. Unless the Department of Defense's (DOD) methodology incorporates all four characteristics of a high-quality, reliable estimate in preparing future cost estimates for the JIAC construction project, it will not be providing decision makers with reliable information.
GAO Summary Assessment of the Air Force's February 2015 JIAC Cost Estimate
|
Characteristic |
GAO Assessment |
|
Comprehensive |
Partially Met |
|
Well documented |
Partially Met |
|
Accurate |
Partially Met |
|
Credible |
Minimally Met |
Source: GAO analysis of DOD information. I GAO-17-29
Note: “Fully met” means the agency provided complete evidence that satisfies the entire criterion. “Substantially met” means the agency provided evidence that satisfies a large portion of the criterion. “Partially met” means the agency provided evidence that satisfies about half of the criterion. “Minimally met” means the agency provided evidence that satisfies a small portion of the criterion. “Not met” means the agency provided evidence that does not satisfy any part of the criterion.
After DOD's 2013 decision to consolidate the JIAC at RAF Croughton, DOD organizations conducted multiple reviews in response to congressional interest in Lajes Field, Azores (Portugal) as a potential alternative location for the JIAC, including
U.S. European Command (EUCOM) September 2015 review , a cost comparison and location analysis of RAF Croughton and Lajes Field;
Office of the Secretary of Defense Cost Assessment and Program Evaluation April 2016 cost verification for JIAC , an independent review of EUCOM's September 2015 cost estimates and those developed by the House Permanent Select Committee on Intelligence in July 2015; and
Defense Information Systems Agency May 2016 review on JIAC communications infrastructure requirements , an assessment and comparison of the communications infrastructures at Lajes Field and RAF Croughton with the intelligence mission support requirements, including the communications and technical requirements, for the JIAC.
These reviews produced different cost estimates, in particular for housing and communications infrastructure, because the DOD organizations that developed them relied on different assumptions. DOD officials said that these reviews were not conducted with the same level of rigor as formal cost estimates, because DOD had concluded its analysis of alternatives and no credible new evidence had been produced to indicate the department should revisit its initial decision to consolidate the JIAC at RAF Croughton.
Why GAO Did This Study
DOD's JIAC, which provides critical intelligence support for the U.S. European and Africa Commands and U.S. allies, is currently located in what DOD has described as inadequate and inefficient facilities at RAF Molesworth in the United Kingdom. To address costly sustainment challenges and instances of degraded theater intelligence capabilities associated with the current JIAC facilities, the Air Force plans to spend almost $240 million to consolidate and relocate the JIAC at RAF Croughton in the United Kingdom.
GAO was asked to review analysis associated with consolidating and relocating the JIAC. This report (1) assesses the extent to which DOD's cost estimate for the JIAC consolidation at RAF Croughton aligns with best practices and (2) describes key reviews DOD has conducted since spring of 2013 related to an alternative location for JIAC consolidation. GAO compared the Air Force's February 2015 JIAC cost estimate with GAO best practices for developing federal cost estimates, reviewed key DOD analysis of Lajes Field as a potential alternative location for the JIAC, and interviewed DOD officials.
Recommendations
GAO recommends that DOD update its future construction cost estimates for consolidating the JIAC at RAF Croughton to comply with best practices for cost estimating identified by GAO. DOD did not agree, stating it would waste resources to continue to generate cost estimates once DOD transitions to managing the project with actual cost data. GAO continues to believe that its recommendation is valid, as discussed in this report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of the Assistant Secretary of Defense for Energy, Installations, and Environment | To better enable DOD to provide congressional decision makers with complete and reliable information on the total anticipated costs for the JIAC consolidation efforts, the Office of the Assistant Secretary of Defense for Energy, Installations, and Environment's Basing Office--in coordination with the Office of the Assistant Secretary of the Air Force Installations, Environment and Energy-- should update future construction cost estimates for consolidating the JIAC at RAF Croughton using best practices for cost estimating as identified in the GAO Cost Estimating and Assessment Guide. Specifically, cost estimates for the JIAC consolidation should fully incorporate all four characteristics of a high-quality, reliable estimate. |
In commenting on this report, DOD did not concur with our recommendation. DOD agreed that many components in the GAO Cost Estimating and Assessment Guide are broadly applicable in the decision process leading up to a military construction budget request. However, DOD further stated that once military construction funds are authorized and appropriated by Congress, the department transitions to a project management mode, and it would be a waste of resources to continue to generate cost estimates once they have transitioned to managing project execution using actual cost data. However, as we note in the report, DOD guidance for estimating construction costs, DOD's Unified Facilities Criteria 3-740-05, states that in the MILCON program, construction cost estimates are prepared throughout the planning, design, and construction phases of a construction project to account for the refinement of the project?s design and requirements. The final estimate should document the department?s assessment of the program's most probable cost and ensure that enough funds are available to execute it. As of October 2016, the military construction funds had not been authorized by Congress for the third phase of the JIAC construction project. According to DOD officials, construction is not scheduled to begin until fall of 2017, and the contract has not yet been awarded. Further, the GAO Cost Estimating and Assessment Guide states that regardless of whether changes to the program result from a major contract modification or an overtarget budget, the cost estimate should be regularly updated to reflect all changes. This is also a requirement outlined in OMB's Capital Programming Guide. The purpose of updating the cost estimate is to check its accuracy, defend the estimate over time, and archive cost and technical data for use in future estimates. After the internal agency and congressional budgets are prepared and submitted, it is imperative that cost estimators continue to monitor the program to determine whether the preliminary information and assumptions remain relevant and accurate. Keeping the estimate updated gives decision makers accurate information for assessing alternative decisions. Cost estimates must also be updated whenever requirements change, and the results should be reconciled and recorded against the old estimate baseline. Therefore, we continue to believe that DOD's implementation of our recommendation to update future JIAC cost estimates using the best practices identified in the GAO Cost Estimating and Assessment Guide would assist in ensuring that decision makers have complete and reliable information about costs associated with the JIAC consolidation and as the third phase of the JIAC project is authorized. Implementing our recommendation would also ensure that DOD develops a reliable historical record for the cost of the JIAC that can be used to estimate other similar projects in the future. As of June 2017, the agency had not taken any action to implement this recommendation. As of July 2018, a senior DOD official said that DOD is developing a new analysis of alternatives (AOA) for JIAC consolidation and will use, as appropriate, our AOA best practices. Those best practices include several focused on cost estimation. We have requested information on the extent to which the AOA team will use best practices for cost estimating to update the JIAC consolidation cost estimate. When we confirm what actions DOD has taken, we will provide updated information. As of September 2019, a senior DOD official said that DOD's prior non-concur with our recommendation is still valid for the reasons mentioned above.
|