This is the accessible text file for GAO report number GAO-11-380R entitled 'Defense Acquisitions: Application of Lessons Learned and Best Practices in the Presidential Helicopter Program' which was released on March 25, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-11-380R: United States Government Accountability Office: Washington, DC 20548: March 25, 2011: Congressional Committees: Subject: Defense Acquisitions: Application of Lessons Learned and Best Practices in the Presidential Helicopter Program: In June 2009, following the expenditure of close to $3 billion and a critical Nunn-McCurdy breach[Footnote 1] of the cost growth threshold, the Department of Defense (DOD) terminated[Footnote 2] the Navy's VH- 71 presidential helicopter acquisition program and contract because of cost growth, schedule delays, and projected system performance. The Presidential Helicopter VXX program is a successor Navy program to the terminated VH-71 program acquisition and has been initiated to develop aircraft to replace the current, aging presidential helicopter fleet. The Ike Skelton National Defense Authorization Act for Fiscal Year 2011 (the Act) directed GAO to review and report annually to the congressional defense committees on the VXX program through 2013. [Footnote 3] This is the first of the required GAO reports. It discusses (1) major lessons learned from the terminated VH-71 program that should be applied to the follow-on VXX program and (2) the current acquisition approach of the VXX program and sufficiency of the underlying acquisition plans and related documentation. To identify major VH-71 program lessons learned, we examined the cost, schedule, and performance experience of the terminated VH-71 program through discussions with agency officials and review of reported lessons learned and then assessed how those lessons relate to acquisition best practices. To determine the current status of the VXX program, we examined available program documents and interviewed knowledgeable officials. To assess the sufficiency[Footnote 4] of the program's efforts, we discussed the Navy's plans for the program with knowledgeable officials and obtained and analyzed the one completed VXX program baseline acquisition document, the program's Initial Capabilities Document (ICD),[Footnote 5] to determine whether it incorporates acquisition best practices and addresses DOD's acquisition policy and guidance. We also obtained preliminary information on the ongoing VXX program Analysis of Alternatives (AOA)[Footnote 6] and made preliminary observations on this effort and the terminated VH-71 program's AOA conducted in 2003. We conducted this performance audit from September 2010 to March 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: The Marine Corps's HMX-1 squadron currently utilizes two types of helicopters--the VH-3D and the VH-60N--to carryout the presidential helicopter mission. Over the past several years, numerous modifications and improvements have been made to both aircraft types to incorporate emerging technologies and address new requirements. These improvements have increased the gross weight of the helicopters-- decreasing some aspects of mission capability, for example, range, and severely limiting the ability to incorporate future improvements because of the impact of further weight growth. According to program officials, over time it had become increasingly difficult to accommodate the demands placed on the HMX-1 aircraft in support of presidential requirements. The events following the terrorist attacks on the United States on September 11, 2001, highlighted the need for improved transportation, communication, and security capabilities for presidential support aircraft. As a result, a VXX replacement helicopter program was initiated in April 2002 to begin fielding a new helicopter in the 2011 time frame. A subsequent November 2002 White House memo to the Secretary of Defense articulated a goal of having the new helicopter available earlier, and in response, the Navy developed an accelerated program plan to develop and initially field a new helicopter by the end of 2008. The VXX program was subsequently redesignated the VH-71 program. In January 2005, the Navy entered into a contract with Lockheed Martin Systems Integration to develop the replacement helicopter. By January 2009, after schedule slips and significant cost increases, the Navy reported a Nunn-McCurdy breach of the critical cost growth threshold and in June 2009 terminated the VH-71 program. However, the need for a replacement helicopter remains. An ICD was drafted by the Marine Corps and approved by DOD in August 2009 to start a successor VXX Helicopter Replacement Program. According to program officials, the current projected VH-3D and VH-60N service life will support the development and fielding of the VXX. In the last few years, Congress and DOD have put in place new defense acquisition policy provisions that replace risk with knowledge-- placing greater emphasis on front-end planning and establishing sound business cases for starting programs. For example, both DOD's December 2008 acquisition policy revisions and the Weapon Systems Acquisition Reform Act of 2009 require programs to invest more time and resources in the front end of the acquisition process in line with a knowledge- based acquisition approach.[Footnote 7] As a result, DOD's current acquisition policy and guidance reflect a knowledge-based acquisition framework in line with acquisition best practices GAO has been identifying since the late 1990s. We will use this framework in this report to analyze both the VH-71 and the VXX programs. A knowledge-based acquisition framework involves achieving the right knowledge at the right time--enabling leadership to make informed decisions about when and how best to move into various acquisition phases. In essence, knowledge supplants risk over time. Our best practices work has demonstrated that this building of knowledge consists of information that should be gathered at three critical points over the course of a program. ² Knowledge point 1: Resources and requirements match. Achieving a high level of technology maturity and preliminary system design backed by robust systems engineering by the start of system development is an important indicator of whether this match has been made. This means that the technologies needed to meet essential product requirements have been demonstrated to work in their intended environment. In addition, the developer has completed a preliminary design of the product that shows the design is feasible. If the above conditions are met, a solid business case is established at this point. ² Knowledge point 2: Product design is stable. This point occurs when a program determines that a product's design will meet customer requirements, as well as cost, schedule, and reliability targets. A best practice is to achieve design stability at the system-level critical design review, usually held midway through system development. Completion of at least 90 percent of engineering drawings at this point provides tangible evidence that the product's design is stable, and a prototype demonstration shows that the design is capable of meeting performance requirements. ² Knowledge point 3: Manufacturing processes are mature. This point is achieved when it has been demonstrated that the developer can manufacture the product within cost, schedule, and quality targets. A best practice is to ensure that all critical manufacturing processes are in statistical control--that is, they are repeatable, sustainable, and capable of consistently producing parts within the product's quality tolerances and standards--at the start of production. Summary: Several lessons learned from the acquisition strategy and eventual termination of the VH-71 program apply to the VXX program. For example, had the VH-71 program followed acquisition best practices and conducted early systems engineering, it could have led to a feasible, stable preliminary design ideally before development start. In turn, a stable, early design allows for more accurate program cost estimates and a better foundation for sufficient funding commitments. Instead, it began without completing systems engineering until well after development start. As a consequence, it never achieved design stability and experienced significant cost and schedule problems in development. Its cost estimates doubled--from about $6.5 billion at development start in 2005 to about $13 billion when terminated in 2009. More than good systems engineering is necessary, however. A key to successful development is the ability to make early trade-offs either in the design of a product or the customer's expectations to avoid outstripping the resources available for product development. The VH-71 program was not afforded room needed to pursue these needed trade-offs. Stringent performance requirements (some with no flexibility) were laid out for the system prior to the start of development and did not appear to involve significant consideration of trade-offs of cost, performance, and schedule negotiated between the customer and the developer. The VH-71 program's experience validated the need to execute a knowledge-based acquisition process with discipline, confirming the danger of not replacing risk with knowledge earlier in the acquisition process. VXX program officials seem to understand this lesson learned from the VH-71 program and appear to be establishing a knowledge-based acquisition process emphasizing early systems engineering. One of the primary lessons they learned from the VH-71 program's experience is that there must be an early, solid business case with a rational balance between requirements, cost, and schedule. To accomplish this, they have stated that a rigorous four-phase systems engineering and technical review process will be used. Early VXX program efforts appear to reflect the intent to pursue a knowledge-based acquisition. The VXX program is currently in the materiel solution analysis phase of the acquisition process, and an ICD has been developed to formally document the capabilities required to perform the defined mission, the specific capability gaps that exist, and the need to address them. Our review of the VXX program's ICD indicates that it addresses all three of these areas and also appears to align with acquisition best practices. An AOA has been initiated but not yet completed. According to DOD officials, it will be more robust than the AOA developed for the VH-71 program. The program is in the earliest stages of development, still developing a business case to launch product development. We will, as you have requested, assess and report on a wide range of VXX program activities moving forward. Throughout, we will be looking for the implementation of a knowledge-based acquisition through application of key best practice process controls. Lesson Learned from the VH-71 Program: The VH-71 program's failure to follow acquisition best practices was a critical factor in the program's poor performance that led to its ultimate termination. It started with a faulty business case, did not perform appropriate systems engineering analysis to gain knowledge at the right times, and failed to make necessary trade-offs between resources and requirements even after years of development. Because of these failures, the program was unable to achieve stable design and experienced significant cost growth and schedule delays before being terminated in 2009. The Program Began with a High-Risk Business Case: The VH-71 program was knowingly initiated with a high-risk business case. To accelerate the program's initial delivery of helicopters from 2011 to 2008, the Navy adopted a two-step acquisition approach and initiated production at the same time it began development.[Footnote 8] Prior to the start of development, a March 2004 Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (OUSD(AT&L)) Defense Acquisition Program Support Assessment of the program concluded that the program had a high-risk schedule because of concurrent design and production efforts. However, the Office of the Secretary of Defense (OSD) recognized and accepted the program's risk and strategy in its January 2005 Acquisition Decision Memorandum approving the program's entry into development. Concurrency and lack of systems engineering knowledge to achieve a match between required capabilities and resources contributed to the cost overruns and schedule delays experienced by the program prior to its termination in 2009. Systems Engineering Knowledge Was Not Available: The risk of the VH-71 program's business case was even higher than anticipated because early systems engineering wasn't done when it should have been to be in accord with acquisition best practices. As we have previously reported,[Footnote 9] a primary reason for cost and schedule problems is too many technical unknowns and insufficient knowledge about performance and production risks. If this early systems engineering is not performed, significant cost increases can occur as the system's requirements become better understood by the government and contractor. The VH-71 program's experience exemplified this. System requirement reviews are conducted to ensure that system requirements have been properly identified and that a mutual understanding between the government and the contractor exists. These steps should occur prior to the start of development. However, when the VH-71 program's system requirements review occurred in May 2005--4 months after the start of development--it was found that the contractor's design was significantly noncompliant with what the contract required. In June 2006, the VH-71 program's Defense Acquisition Executive Summary reported that it had taken considerable effort over the prior 14 months to resolve related issues and that good progress had been made in aligning the contractor's design with what the contract required. It also noted, though, that related activities had resulted in delay of technical reviews and contract milestones and additional cost. Having a feasible, stable preliminary design for a weapons program early in the acquisition process is important for lessening risk, both by ensuring that there is a match between resources and requirements and by demonstrating that a product's design can meet customer requirements, as well as cost, schedule, and reliability targets. The VH-71 program did not, however, complete needed systems engineering until well after the start of development for both increments and, in the case of Increment 1, after approval of production as well, as Increment 1 was approved to enter development and production at the same time. OSD approved the Increment 1 entry into development and production in January 2005. A preliminary design review (PDR), which best practices indicate should occur prior to a program entering development, was not completed for the Increment 1 effort until February 2006--13 months after the start of development. OSD also approved Increment 2 development in January 2005. A PDR for Increment 2 had not occurred by the time a stop work order was placed on the Increment 2 effort in December 2007--35 months after the start of Increment 2 development. The stop work order was placed as a result of a concern that the required work would exceed available funding. Similarly, it is a best practice to achieve design stability at a system-level critical design review (CDR), usually held midway through system development--that is, halfway between the start of development and approval to enter production. A CDR for Increment 1 was held in August 2006 but was not completed as significant elements remained to be addressed. Subsequently, an airframe CDR did not occur until February 2007--more than 2 years after the start of Increment 1 development and production. A CDR was scheduled for Increment 2 to occur in fiscal year 2008, but the stop work order for Increment 2 was issued prior to the scheduled completion of the CDR. As knowledge from the successive systems engineering reviews builds, uncertainty and associated risks in the cost estimate decrease. As noted, this knowledge was developed late, if at all, in the VH-71 program and the consequence was cost growth to the degree that at the time the VH-71 program was terminated in June 2009, it had grown from a total estimated cost of about $6.5 billion at the start of development in January 2005 to about $13 billion. The Program Did Not Make Appropriate Trade-offs between Capability Requirements and Allowable Resources: While good systems engineering can identify and inform trade-offs, the customer and developer must be willing to make trade-offs to achieve a successful business case. We have found that key to successful developments was the ability to make early trade-offs either in the design of the product or the customer's expectations to avoid outstripping the resources available for product development. Conversely, as we have found with other programs--such as the Armed Reconnaissance Helicopter program--an unwillingness to make performance trade-offs can contribute to programs being unexecutable, ultimately resulting in their termination. The VH-71 program was not afforded sufficient room needed to pursue needed trade-offs. Stringent performance requirements (some with no flexibility) were laid out for the system prior to the start of development and did not appear to involve significant consideration of trade-offs of cost, performance, and schedule negotiated between the customer and the developer. Rather, as a July 2007 Institute for Defense Analyses[Footnote 10] independent review team reported on the executability of Increment 2: "Unfortunately, the requirements for the program are still not well founded. Initially this reflected a hurried program start, without the foundation-laying analyses and design and requirements tradeoffs, necessary to initiate a program with an acceptable level of stability. Currently, incomplete requirements documentation and poor configuration management presage further instability in the future. The [independent review team] also observed that the two directions given to the program by the Acquisition Decision Memorandum were not followed: * Leadership engagement and clear guidance were imperative for program success. * Disciplined application of systems engineering practices, early configuration control decisions, and strict configuration control were essential." VXX Program Appears Set to Use a Knowledge-Based Acquisition Approach: The VH-71 program experience validated the need to execute a knowledge- based process with discipline and confirmed the danger of not replacing risk with knowledge earlier in the acquisition process. VXX program officials seem to understand this and appear to be moving forward in an attempt to apply these lessons. They have stated that their aim is to establish an initial knowledge-based acquisition business case that will emphasize early systems engineering; mature technologies; an incremental, knowledge-based approach; and the ability to trade capability for resources. The program is currently in its earliest stages, nearing completion of the materiel solution analysis phase, and is likely still years away from establishing its business case. Therefore, we do not have much acquisition documentation to review beyond the ICD, information on the preliminary acquisition planning, and AOA plans. VXX program management indicated that one of the primary lessons learned from the VH-71 program experience is that DOD must establish and maintain a solid business case, where a balance exists between requirements, cost, and schedule that results in an executable program with acceptable risk. To accomplish this, the program intends to use a rigorous four-phase systems engineering and technical review process constituting a more knowledge-based acquisition approach. This process differs greatly from the process followed during the VH-71 program. According to VXX program officials, the VXX program's systems engineering and technical review process will begin earlier during the technology development phase (pre-Milestone B). VXX program officials stated that they plan to enter the systems development phase with a preliminary technical design and possibly an early prototype of subsystems. Early VXX program efforts appear to reflect the intent to pursue a best practices-aligned knowledge-based acquisition. The VXX program is currently in the pre-Milestone A[Footnote 11] phase of the acquisition process. An ICD has been approved and an AOA is currently being finalized; both are required for Milestone A. After termination of the VH-71 program contract, DOD conducted an analysis to determine if a nonmateriel solution existed to fill the identified gap in the presidential helicopter program's vertical lift capability. DOD determined that there were no nonmaterial approaches that would significantly improve or resolve the capability gaps.[Footnote 12] It was determined that a materiel solution must be pursued, and as a result, an ICD for the VXX program was developed in August 2009. According to the Chairman of the Joint Chiefs of Staff instruction that guides the process that develops ICDs, an ICD is a formal capability assessment by a military service, defense agency, or other sponsor and it formally documents (1) the capabilities required to perform the defined mission, (2) the specific capability gap or set of gaps that exist, and (3) the need to address the capability gap(s). [Footnote 13] Our review of the VXX program's ICD indicates that it addressed all three of these areas. The ICD and its intended use also appear to align with acquisition best practices. Unlike the requirements document that led to the VH-71 program, the VXX program's ICD identifies performance attributes as "preferred values" rather than key performance parameters. Program officials indicated that the use of preferred values instead of firm requirements will allow the program room to do trade-offs needed to achieve a match between the users' requirements and available resources before development begins-- an acquisition best practice. According to a DOD official, the Navy expects to issue the results of its AOA in the second quarter of 2011. According to DOD officials, it will be more robust than the AOA used to support the VH-71 program. While we cannot assess its sufficiency until it is complete, statements by DOD officials on the nature of the AOA effort suggest the pursuit of an acquisition approach in line with best practices. Plans are for the AOA to analyze in detail the estimated cost and effectiveness of a range of potential materiel solutions to support the need. In the past, we have reported that many AOAs do not effectively consider a broad range of alternatives for addressing a need or assess technical and other risks associated with each alternative.[Footnote 14] Without a sufficient comparison of alternatives and focus on technical and other risks, AOAs may identify solutions that are not feasible and decision makers may approve programs based on limited knowledge. While many factors can affect cost and schedule outcomes, we have found that programs that had a limited assessment of alternatives tended to have poorer outcomes than those that had more robust AOAs. When the AOA is issued, we will assess it for robustness--the range of alternatives it considers, its depth of analysis, and its consideration of trade-offs. As the program progresses, we will continue to assess and report its performance as directed in the Ike Skelton National Defense Authorization Act for Fiscal Year 2011 (the Act). Figure 1 illustrates the acquisition process and where the VXX program currently stands in that process, and identifies some of the key program documents we will review in the future and the best practices criteria that the program should meet as it progresses. We will use this as a template for future reviews of the program. Figure 1: Alignment of DOD's Acquisition Process, Key Program Documents, and Best Practices: [Refer to PDF for image: illustration] DOD Acquisition process: Materiel solution analysis: * Materiel development decision; * Current VXX status; * Key program documents: - ICD. Milestone A: Milestone A documents: AOA; Component cost estimate; Systems engineering plan; Technology development strategy; Test and evaluation strategy. DOD Acquisition process: Technology development: * PDR; Best practices knowledge-based acquisition model: Knowledge Point 1: Technologies, time, funding and other resources match customer needs. Decision to invest in product development. Key steps: * PDR completed; * Technologies demonstrated to high levels; * Incremental acquisition strategy in place; * Knowledge-based cost estimate. Milestone B: Development start; Milestone B documents: Acquisition strategy; Capability development document; Independent cost estimate; Technology readiness assessment (TRA); Test and evaluation master plan. DOD Acquisition process: Engineering and manufacturing development: Integrated system design: CDR; Knowledge Point 2: Design is stable and performs as expected. Decision to start building and testing production representative prototypes. Key steps: * System-level CDR and subsystem design reviews completed; * Ninety percent of engineering drawings released; * Integrated system prototype demonstrated; * Critical manufacturing processes identified. DOD Acquisition process: Engineering and manufacturing development: Capability and manufacturing demo: Knowledge Point 3: Production meets cost, schedule, and quality targets. Decision to produce first units for customer. Key steps: * Production-representative prototype demonstrated in intended environment; * Manufacturing processes in control; * Product reliability demonstrated via production-representative prototype testing. Milestone C: Production start. Milestone C documents: Acquisition strategy; Capability production document; Independent cost estimate; TRA; Test and evaluation master plan. DOD Acquisition process: Production. Source: GAO analysis of DOD's acquisition process and GAO's best practices. [End of figure] By using this framework, we will address the issues identified in the Act. Specifically: * the extent to which the program is meeting development and procurement cost, schedule, performance, and risk mitigation goals; * the progress of developmental and operational testing of aircraft and plans for correcting deficiencies in aircraft performance, operational effectiveness, reliability, suitability, and safety; * the program procurement plans, production results, and efforts to improve manufacturing efficiency and supplier performance; * the program's acquisition strategy, including whether it is in accord with acquisition best practices and DOD's acquisition policy and regulations; * risk assessments of its integrated master schedule and test and evaluation master plan; and: * our assessment of the sufficiency of the program's ICD (if revised), AOA, capabilities development document, and a systems requirement document (if produced). Overall, we will assess the use of acquisition best practices throughout the VXX program's acquisition through management's adherence to key best practice model controls. Those controls are outlined in the table in enclosure I. Agency Comments: DOD provided technical comments on the information in this report, which GAO incorporated as appropriate, but declined to provide additional comments. We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Under Secretary of Defense for Acquisition, Technology and Logistics; and the Secretary of the Navy. This report also is available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions on the matters covered in this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in enclosure II. Signed by: Michael J. Sullivan, Director: Acquisition and Sourcing Management: Enclosures - 2: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel K. Inouye: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable C. W. Bill Young: Chairman: The Honorable Norman D. Dicks: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Enclosure I: Best Practices Model Controls at Key Acquisition Process Points: Milestone A: Occurs as programs begin the technology development phase; Criteria: * Complete robust Analysis of Alternatives. Milestone B (knowledge point 1): Occurs as programs begin the engineering and manufacturing development phase (Milestone B). Match exists between requirements and resources. Completed when technologies needed to meet essential product requirements have been demonstrated to work in their intended environments and the producer has completed a preliminary design of the product; Criteria: * Demonstrate high technology readiness levels; * Ensure that product requirements are informed by the systems engineering process; * Establish cost and schedule estimates for the product based on knowledge from preliminary design using systems engineering tools; * Complete preliminary design review; * Conduct decision review for program launch. Critical design review (knowledge point 2): Occurs at the critical design review between integration and demonstration. Completed when design is stable and has been demonstrated through prototype testing and 90 percent of engineering drawings are releasable to manufacturing organizations; Criteria: * Complete 90 percent of design drawings; * Complete subsystem and system design reviews; * Demonstrate with prototype that design meets requirements; * Obtain stakeholder concurrence that drawings are complete and producible; * Complete failure modes and effects analysis; * Identify key system; characteristics; * Identify critical manufacturing processes; * Establish reliability targets and growth plan based on demonstrated reliability rates of components and subsystems; * Conduct design review to enter system demonstration. Milestone C (knowledge point 3): Occurs at low-rate initial production commitment. Completed when product is ready to be manufactured within cost, schedule, and quality targets and all key manufacturing processes are under statistical control and product reliability has been demonstrated; Criteria: * Demonstrate manufacturing processes; * Build production-representative prototypes; * Test production-representative prototypes to achieve reliability goal; * Test production-representative prototypes to demonstrate the product in a realistic environment; * Collect statistical process control data; * Demonstrate that critical processes are capable and under statistical control; * Conduct decision review to begin production. Source: GAO. [End of table] [End of section] Enclosure II: GAO Contact and Staff Acknowledgments: GAO Contact: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov: Staff Acknowledgments: Key contributors to this report were Bruce H, Thomas, Assistant Director; Jerry Clark, Analyst-in-Charge; Marie P. Ahearn; Michael Aiken; Kenneth Patton; Don Springman; Robert Swierczek; and J. Andrew Walker. [End of section] Related GAO Products: Defense Acquisitions: Strong Leadership Is Key to Planning and Executing Stable Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-10-522]. Washington, D.C.: May 6, 2010. Defense Acquisitions: Assessments of Selected Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-10-388SP]. Washington, D.C.: March 30, 2010. Defense Acquisitions: Many Analyses of Alternatives Have Not Provided a Robust Assessment of Weapon System Options. [hyperlink, http://www.gao.gov/products/GAO-09-665]. Washington, D.C.: September 24, 2009. Defense Acquisitions: Measuring the Value of DOD's Weapon Programs Requires Starting with Realistic Baselines. [hyperlink, http://www.gao.gov/products/GAO-09-543T]. Washington, D.C.: April 1, 2009. Defense Acquisitions: Assessments of Selected Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-09-326SP]. Washington, D.C.: March 30, 2009. Defense Acquisitions: DOD Must Balance Its Needs with Available Resources and Follow an Incremental Approach to Acquiring Weapon Systems. [hyperlink, http://www.gao.gov/products/GAO-09-431T]. Washington, D.C.: March 3, 2009. GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. Washington, D.C.: March 2009. Defense Acquisitions: Perspectives on Potential Changes to Department of Defense Acquisition Management Framework. [hyperlink, http://www.gao.gov/products/GAO-09-295R]. Washington, D.C.: February 27, 2009. Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon System Program Outcomes. [hyperlink, http://www.gao.gov/products/GAO-08-619]. Washington, D.C.: July 2, 2008. Defense Acquisitions: Assessments of Selected Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-08-467SP]. Washington, D.C.: March 31, 2008. Defense Acquisitions: Assessments of Selected Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-07-406SP]. Washington, D.C.: March 30, 2007. Defense Acquisitions: Assessments of Selected Major Weapon Programs. [hyperlink, http://www.gao.gov/products/GAO-06-391]. Washington, D.C.: March 31, 2006. Defense Acquisitions: Major Weapon Systems Continue to Experience Cost and Schedule Problems under DOD's Revised Policy. [hyperlink, http://www.gao.gov/products/GAO-06-368]. Washington, D.C.: April 13, 2006. Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes. [hyperlink, http://www.gao.gov/products/GAO-06-110]. Washington, D.C.: November 30, 2005. Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes. [hyperlink, http://www.gao.gov/products/GAO-02-701]. Washington, D.C.: July 15, 2002. Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes. [hyperlink, http://www.gao.gov/products/GAO-01-288]. Washington, D.C.: March 8, 2001. Best Practices: A More Constructive Test Approach Is Key to Better Weapon System Outcomes. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-199]. Washington, D.C.: July 31, 2000. Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes. [hyperlink, http://www.gao.gov/products/GAO/NSIAD-99-162]. Washington, D.C.: July 30, 1999. [End of section] Footnotes: [1] 10 U.S.C. § 2433 establishes the requirement for the Department of Defense (DOD) to prepare unit cost reports on major defense acquisition programs or designated major defense subprograms. If a program exceeds cost growth thresholds specified in the law, this is commonly referred to as a Nunn-McCurdy breach. DOD is required to report these breaches to Congress, and in certain circumstances, DOD must reassess the program and submit a certification to Congress in order to continue the program, in accordance with 10 U.S.C. § 2433a. The Nunn-McCurdy breach experienced by the VH-71 program was estimated to represent program acquisition unit cost (PAUC) growth of more than 50 percent above the original acquisition program baseline--a level that would have required certification to Congress for continuation of the program if the program had not been terminated. PAUC represents the expected acquisition cost for each unit procured, as determined by dividing the sum of a program's total program development, procurement, and system-specific military construction costs by the number of fully configured end items to be procured. [2] A memorandum canceling the program was issued in May 2009. Subsequently, a notice terminating the contract was sent to the contractor in June 2009. In accordance with the termination clause, the contractor was required to submit a termination settlement proposal within 1 year. The contractor submitted its termination proposal in May 2010. The termination proposal is currently being audited and a final settlement is expected by the end of fiscal year 2011. [3] Pub. L. No. 111-383 § 233. [4] We have defined sufficiency as meaning objective and in accord with acquisition best practices and DOD acquisition policy and guidance. [5] A sponsor, usually a military service, submits a capability proposal called an ICD through DOD's requirements determination process--the Joint Capabilities Integration and Development System. An ICD identifies the existence of capability gap(s), the operational risks associated with the gap(s), and a recommended solution or preferred set of solutions for filling the gap(s). [6] An AOA is an evaluation of the performance, effectiveness, suitability, and estimated costs of alternative systems to meet a capability. [7] In December 2008, DOD revised its acquisition instruction-- Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System. The Weapon Systems Acquisition Reform Act of 2009, Pub. L. No. 111-23, was enacted on May 22, 2009. [8] If done correctly, the use of an incremental approach to development is a best practice that our work has shown enables organizations to achieve a match between needs and resources. Under this approach, basic requirements are achieved first, with additional capabilities planned for future generations of the product. Because product development is incremental, achieving knowledge is more manageable. As we have reported, commercial companies have found that trying to capture the knowledge needed to stabilize the design of a product with considerable new technical content is an unwieldy task-- especially if the goal is to reduce development cycle times and get the product to the marketplace as quickly as possible. GAO, Best Practices: Using a Knowledge-Based Approach to Improve Weapon Acquisition, [hyperlink, http://www.gao.gov/products/GAO-04-386SP] (Washington, D.C.: January 2004). [9] GAO, Defense Acquisitions: Strong Leadership Is Key to Planning and Executing Stable Weapon Programs, [hyperlink, http://www.gao.gov/products/GAO-10-522] (Washington, D.C.: May 6, 2010). [10] Institute for Defense Analyses, VH-71 Presidential Helicopter Program Assessment, IDA Paper P-4243 (Alexandria, Va., July 2007) (Distribution authorized to U.S. government agencies and their contractors; Specific Authority. Other requests must be referred to OUSD(AT&L).) [11] The VXX Milestone A is currently expected to occur sometime in fiscal year 2011. [12] The analysis typically evaluates military doctrine, organization, training, materiel, leadership and education, personnel, and facilities to try to identify a nonmaterial solution to a capability gap or gaps. [13] Chairman of the Joint Chiefs of Staff Instruction 3170.01G, Joint Capabilities Integration and Development System (2009), encl. A, 2(d). When the Joint Requirements Oversight Council approves an ICD, it validates the capabilities required to perform the mission as defined, the gaps in capabilities along with their priorities and operational risks, and the need to address the capability gaps. [14] GAO, Defense Acquisitions: Many Analyses of Alternatives Have Not Provided a Robust Assessment of Weapon System Options, [hyperlink, http://www.gao.gov/products/GAO-09-665] (Washington, D.C.: Sept. 24, 2009). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: