This is the accessible text file for GAO report number GAO-15-210R entitled 'Missile Defense: Cost Estimating Practices Have Improved, and Continued Evaluation Will Determine Effectiveness' which was released on December 12, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-15-210R: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: December 12, 2014: Congressional Committees: Missile Defense: Cost Estimating Practices Have Improved, and Continued Evaluation Will Determine Effectiveness: Since 2002, the Missile Defense Agency (MDA) has been charged with developing and fielding the Ballistic Missile Defense System (BMDS). This highly complex group of systems is made up of a diverse collection of land-, sea-, and space-based assets, including sensors, interceptors, and battle management located around the globe, with the purpose of detecting, tracking, and defeating enemy ballistic missiles. For over a decade, we have annually reported on MDA's progress and challenges in developing and fielding BMDS capabilities, as well as transparency, accountability, and oversight issues.A list of our reports can be found in the enclosure. The Department of Defense has spent approximately $105 billion since 2002 on MDA's BMDS and plans to spend an additional $37.5 billion from fiscal years 2015 through 2019. However, since we began annual reporting on missile defense in 2004, we have been unable to assess overall progress on cost because the cost information MDA reported over the years has been incomplete, inconsistently reported, or lacked sufficient quality for review. In an effort to establish a means for us to evaluate MDA's cost, we have made a number of recommendations-- and Congress has passed a number of laws--that are related to the establishment of baselines for the expected cost of certain elements of the BMDS and to reporting deviations from the baseline as MDA's programs progress. MDA created those baselines in 2010, and in 2011 we found that the cost estimates underlying the baselines were not high- quality, according to best practices contained in the GAO Cost Estimating and Assessment Guide.[Footnote 1] Further, we concluded that cost progress cannot be independently evaluated until MDA establishes baselines that are supported by reliable, high-quality cost estimates. The National Defense Authorization Act for Fiscal Year 2014 mandated that MDA submit a report to the congressional defense committees by December 26, 2014, describing, among other things, the actions it has taken to improve the quality of its cost estimates, the schedule for any remaining actions, and how those actions compare to the best practices and cost-estimation guidelines recommended by GAO.[Footnote 2] In addition, the agency's report must also include GAO's views on its contents. This report includes our assessment of MDA's October 2014 draft report and the extent to which MDA's past and planned actions are in line with GAO best practices. Specifically, our assessment focuses on four key actions MDA has taken to improve the quality of its cost estimates: (1) releasing a June 2012 Cost Estimating and Analysis Handbook, (2) performing internal cost reviews, (3) assembling an independent cost assessment team, and (4) receiving independent cost estimates from the Office of the Secretary of Defense for Cost Assessment and Program Evaluation (CAPE). This report is being published prior to the release of MDA's report so that it can be included as mandated in MDA's final report to congressional defense committees. To assess the actions MDA has taken to improve the quality of its cost estimates, we focused on four key areas that form the basis of MDA's efforts to improve its cost estimates. First, we evaluated MDA's cost handbook by analyzing the content to see how well it compares to the best practices in GAO's guide. Second, while MDA provided cost review scores for all four of GAO's characteristics of a high quality estimate, we focused our assessment on the comprehensive characteristic.[Footnote 3] We examined MDA's comprehensive scores for eleven cost estimates and determined whether we concurred or did not concur with MDA's rating based on their rationale and supporting documentation.[Footnote 4] We reviewed the comprehensive characteristic because, according to GAO's guide, if the cost estimate is not comprehensive then it cannot fully meet the other best practice characteristics. We also met with MDA cost estimators who performed these cost reviews to discuss our initial findings and obtain additional information regarding the rationale for MDA's comprehensiveness scores. Third, we evaluated the effectiveness of MDA's independent cost assessment team by meeting with the team's deputy director, reviewing its completed reports, and discussing the team's processes and products. Lastly, we reviewed recent CAPE independent cost assessment results for MDA cost estimates to see how they compared with MDA's cost estimates and held discussions with CAPE cost estimating experts. We conducted this performance audit from October 2014 to December 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusion based on our audit objectives. Summary: MDA has taken positive steps to improve the quality of its cost estimates, and continued evaluation will determine the effectiveness of these steps. In 2011, we reviewed MDA cost estimates and found that none of the estimates we reviewed substantially met the criteria for any of the four characteristics called out in GAO's cost guide to be considered high quality.[Footnote 5] MDA has since taken steps that are in-line with GAO best practices including: releasing an internal cost estimating handbook, performing internal cost reviews to evaluate the estimates against the standards presented in the handbook, assembling an independent cost group to perform an additional independent review of the quality of MDA's estimates, and receiving independent cost estimates from an external organization. Our evaluation of MDA's cost handbook shows that it is well aligned with the standards in the GAO cost guide. In addition, GAO concurred with 29 out of 33 MDA cost review scores for one of the four GAO characteristics of a high quality cost estimate--comprehensiveness-- which indicates that the agency can effectively assess the comprehensiveness of its estimates. However, the independent cost assessment team MDA assembled in January 2013 for an additional review of the quality of its cost estimates is still defining its processes and had completed reviews on only one program's estimate at the time of our review. Therefore, it is too early to determine the effect that group will have on the quality of future estimates. Lastly, while independent cost assessments and estimates for MDA programs conducted by the CAPE, DOD's principal advisory body for independent cost estimates for DOD acquisition programs, in the past few years have provided some credibility for MDA's estimates, about half of the program baselines presented in the 2014 baseline report still remain unverified. Background: According to the GAO Cost Estimating and Assessment Guide, the cost estimate is a critical element in any acquisition process to help decision makers evaluate resource requirements at milestones and other important decision points.[Footnote 6] Cost estimates establish and defend budgets and drive affordability analysis. Specifically, the guide calls out four characteristics of a high-quality and reliable cost estimate in that they are comprehensive, well documented, accurate, and credible. A cost estimate is considered: * Comprehensive: when it accounts for all possible costs associated with a project, details all cost-influencing ground rules and assumptions, is technically reasonable, is structured in sufficient detail to ensure that costs are neither omitted nor double-counted, and the estimating teams' composition is commensurate with the assignment; * Well-documented: when supporting documentation for the estimate is accompanied by a narrative explaining the process, sources, and methods used to create the estimate and contains the underlying data used to develop the estimate; * Accurate: when the estimate is not overly conservative or too optimistic, and is based on an assessment of the costs most likely to be incurred; and: * Credible: when the estimate has been cross-checked with independent cost estimates, the level of confidence associated with the point estimate--the best guess at the cost estimate given the underlying data--has been identified, and a sensitivity analysis has been conducted. During the sensitivity analysis, the project examined the effect of changing one assumption related to each project activity while holding all other variables constant in order to identify which variable most affects the cost estimate. In 2011, we reviewed MDA cost estimates and found that none of the estimates we reviewed substantially met any of these characteristics to be considered high quality.[Footnote 7] We recommended that MDA take steps to ensure that the cost estimates that underlie the program acquisition baselines are high quality, reliable, and documented to facilitate external review. In addition, Congress required the MDA Director, in the Fiscal Year 2014 National Defense Authorization Act, to submit to the congressional defense committees a report outlining the plans of the Director to improve the quality of those cost estimates.[Footnote 8] According to its October 2014 draft report, MDA has taken a number of steps over the past few years to improve the quality of its cost estimates, including releasing a cost estimating handbook, performing internal cost reviews, assembling an independent cost assessment team, and receiving independent cost estimates from the CAPE. MDA's Internal Cost Estimating Handbook Aligns Well with the Standards Detailed in GAO's Cost Guide: GAO's evaluation of MDA's Cost Estimating and Analysis Handbook shows that it fully or substantially meets all of the standards outlined in the GAO Cost Estimating and Assessment Guide. GAO's cost guide discusses the importance of establishing a disciplined cost estimating process as a basis for the agency to create credible cost estimates. In June 2012, MDA released a Cost Estimating and Analysis Handbook with the primary purpose of standardizing the agency's cost analysis requirements and procedures for the preparation, documentation, and presentation of MDA cost estimates. In developing its handbook, MDA incorporated best practices from the National Aeronautics and Space Administration and GAO as well as comments from the services. GAO's cost guide identifies 12 steps as seen in table 1 that, if followed correctly, should result in reliable and valid cost estimates that management can use for making informed decisions. The chapters in the GAO guide provide supporting information associated with each of those steps. We found that MDA's guide aligns well with the GAO cost guide with eight steps fully meeting the GAO standards and four steps substantially meeting them. Table 1 provides more information on the results of our analysis. Table 1: Summary Assessment of MDA Handbook Compared to GAO Best Practices: GAO Steps of a High Quality Cost Estimating Process: 1: Define Estimates Purpose; GAO Assessment of MDA's Handbook: Substantially Met: MDA's Handbook states that cost estimates are generally used to support the budget making process, but does not discuss varying purposes for the six types of cost estimates listed in the Handbook. 2: Develop the estimating plan; GAO Assessment of MDA's Handbook: Met. 3: Define the program's characteristics; GAO Assessment of MDA's Handbook: Met. 4: Determine the estimating structure; GAO Assessment of MDA's Handbook: Substantially Met: MDA's Handbook discusses the importance of documents that detail and describe the standardized structure for planned work, but does not mention the importance of updating these documents. 5: Identify ground rules and assumptions; GAO Assessment of MDA's Handbook: Substantially Met: MDA's Handbook states that ground rules and assumptions should be documented thoroughly, developed with members of the technical community, approved by management, and tied directly to risk and sensitivity analyses. However, it does not mention the relation of the ground rules and assumptions to historical data. 6: Obtain the data; GAO Assessment of MDA's Handbook: Substantially Met: MDA's Handbook states that historical data is the foundation of a credible cost estimate and discusses the normalization and analysis of data, data collection plan, and ensuring that adequate time is allotted for data collection activities. However, MDA's Handbook states that secondary data has advantages over primary data, which is in contradiction to GAO best practices. 7: Develop the point estimate and compare to an independent cost estimate; GAO Assessment of MDA's Handbook: Met. 8: Conduct a sensitivity Analysis; GAO Assessment of MDA's Handbook: Met. 9: Conduct a risk analysis; GAO Assessment of MDA's Handbook: Met. 10: Document the estimate; GAO Assessment of MDA's Handbook: Met. 11: Present estimate to management; GAO Assessment of MDA's Handbook: Met. 12: Update the estimate; GAO Assessment of MDA's Handbook: Met. Source: GAO analysis of MDA data. GAO-15-210R. Note: Met: MDA provided complete evidence that satisfies the elements of the best practice; Substantially Met: MDA provided evidence that satisfies a large portion of the elements of the best practice; Partially Met: MDA provided evidence that satisfies about half of the elements of the best practice; Minimally Met: MDA provided evidence that satisfies a small portion of the elements of the best practice; and Did Not Meet: MDA provided no evidence that satisfies any of the elements of the best practice. [End of table] GAO Review Indicates MDA Can Effectively Assess Comprehensiveness of Estimates: After examining MDA cost review scores for one of the four GAO characteristics of a high quality cost estimate--comprehensiveness-- GAO concurred with 29 out of 33 of the scores, which indicates that the agency can effectively assess the comprehensiveness of its estimates. In the past few years, MDA has created an internal cost review process to continuously evaluate the development and documentation of its program cost estimates. Given that MDA's cost handbook is well aligned with GAO standards and best practices, the next step is ensuring that the agency is implementing the practices and procedures outlined in its handbook that will work to improve the quality of its estimates. GAO's cost guide states that relying on a standard process that emphasizes pinning down the technical scope of the work, communicating the basis on which the estimate is built, identifying the quality of the data, determining the level of risk, and thoroughly documenting the effort should result in cost estimates that are defensible, consistent, and trustworthy. As part of MDA's cost review process, members of MDA's cost estimating directorate meet annually to compare each of the program estimates to the standards in MDA's handbook. The meeting results in the team assigning scores for each of the characteristics associated with a high quality estimate as outlined in MDA's handbook. Possible scores include "Met", "Substantially Met", "Partially Met", "Minimally Met", and "Not Met". After reviewing MDA's scores for one of the four GAO characteristics of a high quality estimate, comprehensiveness, we determined that we concurred with MDA's rationale and supporting documentation for 29 out of 33, or about 90 percent of MDA's ratings as seen in table 2. According to MDA's handbook, the agency scored the comprehensiveness of the estimates by averaging the scores for three criteria based on how well the estimate: 1. Completely defined the program, reflected the current schedule, and contained technically reasonable assumptions: 2. Captured the complete technical scope of the work to be performed, used a logical work breakdown structure that accounts for all performance criteria and requirements: 3. Included a complete life cycle cost estimate, accounting for development (if any), procurement, operation and support (as applicable), AND disposal: For the MDA score for each of the three criteria for 11 cost estimates, our evaluation resulted in either a "concur" or "does not concur" rating. There were a few cases where we could not concur with MDA's ratings because of some inconsistencies among the cost estimate documentation and areas where the estimate was not complete. Table 2: GAO Review of the MDA Comprehensive Characteristic Ratings for Program Cost Estimates: MDA Cost Estimate: Aegis Ashore; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO did not concur with one of the three criteria for MDA's rating because some of the operations and sustainment costs were not included in the roll-up estimating model. MDA Cost Estimate: Aegis SM-3 Block IB Missiles; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO did not concur with one of the three criteria for MDA's rating because the costs in the estimating model and cost review file were inconsistent. MDA Cost Estimate: Aegis Modernization; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Command, Control, Battle Management and Communications Spiral 8.2; MDA Rating for Comprehensive Characteristic: Partially Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Ground-based Midcourse Defense; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Targets and Countermeasures; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Terminal High Altitude Area Defense; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: AN/TPY-2; MDA Rating for Comprehensive Characteristic: Substantially Met; Brief Explanation of GAO Assessment: GAO did not concur with two of the three criteria for MDA's rating because documents did not include a discussion of or an estimate of disposal costs. MDA Cost Estimate: Upgraded Early Warning Radar; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Facilities[A]; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. MDA Cost Estimate: Global Deployment[A]; MDA Rating for Comprehensive Characteristic: Met; Brief Explanation of GAO Assessment: GAO concurred with all three of the criteria for MDA's rating. Source: GAO analysis of MDA data. GAO-15-210R. [A] MDA provided cost estimates for Facilities and Global Deployment which are portions of the Aegis Ashore cost estimate as presented in the 2014 baseline report. [End of table] MDA's Independent Cost Assessment Team Could Help to Improve the Quality of Estimates but It Is Too Early to Assess: Commensurate with a GAO best practice, MDA also assembled an independent cost assessment team to obtain an additional review of the quality of its cost estimates; however, the team has not yet defined its processes or produced enough results for us to assess. According to GAO's guide, an independent cost assessment team may be used to determine whether the cost estimate reflects the program of record by providing an evaluation of a cost estimate's quality and accuracy, looking specifically at a program's technical approach, risk, and acquisition strategy to ensure that the program's cost estimate captures all requirements. MDA's independent cost assessment team, established in January 2013, collects MDA's cost estimates and supporting documentation, and then uses the MDA handbook as its reference point to evaluate how well the estimates are meeting the standards established in the handbook. The results of the independent group's assessment are included in a detailed report, with descriptions for why the estimates met or failed to meet each criteria and subcriteria related to the characteristics of a high quality estimate as outlined in the MDA handbook. At the time of our review, the independent cost team had only produced two detailed reports on its findings related to one program's 2013 estimate. According to the team's Deputy Director, it was also currently drafting a guide detailing the team's procedures and reporting format after incorporating lessons learned from its 2013 reviews. The team plans to finalize its guide by January 2015 and complete a review for all of the program estimates in the annual baseline report by the end of 2015. CAPE Independent Cost Estimates Validate Some MDA Programs, but Costs for About Half of the Programs Remain Unverified: While the CAPE has conducted independent cost assessments for MDA programs in the past few years and its estimates have provided some credibility for MDA's estimates, about half of the cost baselines that MDA presents in its annual baseline report to Congress still remain unverified. One of the criteria for a credible cost estimate is having an independent cost estimate conducted by an organization independent of the acquisition chain of command. According to GAO's guide, an independent cost estimate is based on the same detailed technical and procurement information used to make the program's estimate. The independent estimate might cover a program's entire life cycle, one program phase, or one high-value, highly visible, or high-interest item within a phase. Because the team performing the independent cost estimate is independent, it provides an unbiased test of whether the program office cost estimate is reasonable. It is also used to identify risks related to budget shortfalls or excesses. Other DOD major defense acquisition programs are required to have an independent cost estimate prepared or approved by the CAPE which consists of full life-cycle cost--all costs of development, procurement, military construction, and operations and support, without regard to funding source or management control--before advancing through major milestones.[Footnote 9] Because of the flexibilities granted to MDA in 2002, elements of the BMDS are not yet required to obtain independent cost estimates, and while it has not required its programs to do so, the agency has requested some independent cost estimates from the CAPE.[Footnote 10] In 2010, we recommended that MDA obtain CAPE independent cost estimates for its program baselines.[Footnote 11] MDA concurred with our recommendation and has since received CAPE independent cost assessments and estimates for three of the nine MDA programs presented in the 2014 baseline report.[Footnote 12] The CAPE is also currently finalizing its results for an additional independent cost assessment for a portion of the costs of missiles for a fourth MDA program, the Terminal High Altitude Area Defense. In addition, CAPE memos and briefings from 2012 and 2014, which present comparisons of its results to MDA's estimates, show that the estimates are close and appear to provide validation for many areas of MDA's estimates. According to the GAO cost guide, independent cost estimates for DOD programs are usually higher, and more accurate, than baseline estimates. Thus, if a program cost estimate is close to independent cost estimate results, one can be more confident that it is accurate and more likely to result in funding at a reasonable level. About half of MDA programs still remain unverified. Five programs or program components presenting resource baselines in the 2014 baseline report still have not received CAPE independent cost estimates.[Footnote 13] By the time the CAPE completes its assessment for the Terminal High Altitude Area Defense missiles, the agency will have received independent results for about half of its programs, representing less than 25 percent of the total baselined costs in MDA's 2014 baseline report. Concluding Observations: MDA has made a considerable effort to improve its cost estimating practices since our initial review in 2011, but time will tell as MDA's internal cost review and its independent cost assessment teams continue to evaluate how well the agency implements recommendations from prior reviews to improve future estimates. Our assessment of MDA's cost reviews indicates, and continued evaluation by MDA's independent cost assessment team will provide additional assurance, that the agency is able to identify areas where the cost estimate follows its guide and where correction is needed. MDA's establishment of internal cost reviews and an independent cost assessment team are best practices that will enable the agency to assess how well it is correcting those deficiencies over time and doing the real work of improving the quality of the estimates. Additionally, MDA has made progress in obtaining independent estimates from the CAPE for its baselined programs, and additional estimates from the CAPE will further work to validate MDA estimates and provide more credibility to MDA's cost estimating results. Agency Comments: We are not making any recommendations in this report because we have previously recommended that the Secretary of Defense ensure that MDA: * take steps to ensure cost estimates for each acquisition baseline are high quality, reliable, and documented to facilitate external review; * obtain CAPE independent cost estimates for each baseline. We provided a draft of this report to MDA for comment. The agency provided technical comments which were incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, and to the Director, MDA. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report include David B. Best, Assistant Director, LaTonya Miller, Assistant Director, Brian Bothwell, Meredith Allen Kimmett, Joseph W. Kirschbaum, Jennifer Leotta, Kenneth E. Patton, Karen Richey, and Alyssa Weir. Signed by: Cristina Chaplain: Director, Acquisition and Sourcing Management: Enclosure: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard J. Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Rodney Frelinghuysen: Chairman: The Honorable Pete Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Enclosure: GAO's Annual Reports on Missile Defense: Report Number: [hyperlink, http://www.gao.gov/products/GAO-04-409]; Report Date: Apr. 2004; Report Title: Missile Defense: Actions Are Needed to Enhance Testing and Accountability. Report Number: [hyperlink, http://www.gao.gov/products/GAO-05-243]; Report Date: Mar. 2005; Report Title: Defense Acquisitions: Status of Ballistic Missile Defense Program in 2004. Report Number: [hyperlink, http://www.gao.gov/products/GAO-06-327]; Report Date: Mar. 2006; Report Title: Defense Acquisitions: Missile Defense Agency Fields Initial Capability but Falls Short of Original Goals. Report Number: [hyperlink, http://www.gao.gov/products/GAO-07-387]; Report Date: Mar. 2007; Report Title: Defense Acquisitions: Missile Defense Acquisitions Strategy Generates Results but Delivers Less at a Higher Cost. Report Number: [hyperlink, http://www.gao.gov/products/GAO-08-448]; Report Date: Mar. 2008; Report Title: Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals. Report Number: [hyperlink, http://www.gao.gov/products/GAO-09-338]; Report Date: Mar. 2009; Report Title: Defense Acquisitions: Production and Fielding of Missile Defense Components Continue with Less Testing and Validation Than Planned. Report Number: [hyperlink, http://www.gao.gov/products/GAO-10-311]; Report Date: Feb. 2010; Report Title: Defense Acquisitions: Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach. Report Number: [hyperlink, http://www.gao.gov/products/GAO-11-372]; Report Date: Mar. 2011; Report Title: Missile Defense: Actions Needed to Improve Transparency and Accountability. Report Number: [hyperlink, http://www.gao.gov/products/GAO-12-486]; Report Date: Apr. 2012; Report Title: Missile Defense: Opportunity Exists to Strengthen Acquisitions by Reducing Concurrency. Report Number: [hyperlink, http://www.gao.gov/products/GAO-13-432]; Report Date: Apr. 2013; Report Title: Missile Defense: Opportunity to Refocus on Strengthening Acquisition Management. Report Number: [hyperlink, http://www.gao.gov/products/GAO-14-351]; Report Date: Apr. 2014; Missile Defense: Mixed Progress in Achieving Acquisition Goals and Improving Accountability. Source: GAO. GAO-15-210R. Note: Since 2002, GAO has prepared annual assessments of MDA's progress toward its acquisition goals according to congressional mandates included in the National Defense Authorization Act for Fiscal Year 2002, Pub. L. No. 107-107, § 232(g) (2001); Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, § 233 (2004); National Defense Authorization Act for Fiscal Year 2006, Pub. L. No. 109-163, § 232; John Warner National Defense Authorization Act for Fiscal Year 2007, Pub. L. No. 109-364, § 224 (2006); National Defense Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, § 225; and National Defense Authorization Act for Fiscal Year 2012, Pub. L. No. 112-81, § 232 (2011). [End of table] [End of section] Footnotes: [1] GAO, Missile Defense: Actions Needed to Improve Transparency and Accountability, [hyperlink, http://www.gao.gov/products/GAO-11-372] (Washington, D.C.: Mar. 24, 2011) and GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). [2] National Defense Authorization Act for Fiscal Year 2014, Pub. L. No. 113-66, § 231(c) (2013). The cost estimates that are discussed in DOD's report are those cost estimates that support cost baselines presented in MDA's annual baseline report required by 10 U.S.C. § 225. This baseline report, known as MDA's BMDS Accountability Report, presents the current estimate of the BMDS program baselines which include resource (cost) and schedule baselines. [3] The GAO cost guide calls out four characteristics of a high- quality and reliable cost estimate in that they are comprehensive, well documented, accurate, and credible. A cost estimate is considered comprehensive when it accounts for all possible costs associated with a project, details all cost-influencing ground rules and assumptions, is technically reasonable, is structured in sufficient detail to ensure that costs are neither omitted nor double-counted, and the estimating teams' composition is commensurate with the assignment. [4] MDA provided its review of 11 cost estimates that support nine programs or program components presenting baselines in the annual baseline report. The nine program or program components in the baseline report are: Aegis Ashore; Aegis SM-3 Block IB missiles and associated software; Aegis Modernization; Command, Control, Battle Management and Communications Spiral 8.2; Ground-based Midcourse Defense; Targets and Countermeasures; Terminal High Altitude Area Defense; Army Navy/Transportable Radar Surveillance - Model 2, and Upgraded Early Warning Radar. The agency provided two additional cost estimates that are included in its baselined costs for Aegis Ashore's Facilities and Global Deployment. [5] [hyperlink, http://www.gao.gov/products/GAO-11-372]. [6] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. [7] [hyperlink, http://www.gao.gov/products/GAO-11-372]. [8] Pub. L. No. 113-66, § 231(c). [9] 10 U.S.C. § 2434 requires an independent cost estimate of the full life-cycle cost of the program before a major defense acquisition program can advance into system development and demonstration (now known as engineering and manufacturing development) or production and deployment. [10] Because the BMDS has not yet formally entered the DOD acquisition cycle, the requirement at 10 U.S.C. § 2434, which is triggered by the phases of the cycle, does not yet apply. [11] GAO, Defense Acquisitions: Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach, [hyperlink, http://www.gao.gov/products/GAO-10-311] (Washington, D.C.: Feb. 25, 2010). [12] In 2012, the CAPE completed an independent cost assessment which included the CAPE's estimates for Aegis Ashore, Aegis SM-3 Block IB and associated software, and Aegis Modernization. In addition, in 2014, it completed an independent cost assessment for a portion of the costs of Aegis SM-3 Block IB missiles. [13] These five programs or program components include Command, Control, Battle Management and Communications Spiral 8.2, Ground-based Midcourse Defense, Targets and Countermeasures, Upgraded Early Warning Radars, and AN/TPY-2. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]