This is the accessible text file for GAO report number GAO-14-522R entitled 'Department of Defense's Waiver of Competitive Prototyping Requirement for the Air Force's B-2 Defensive Management System Modernization Program' which was released on April 22, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-14-522R: United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: April 22, 2014: Congressional Committees: Department of Defense's Waiver of Competitive Prototyping Requirement for the Air Force's B-2 Defensive Management System Modernization Program: The Weapon Systems Acquisition Reform Act of 2009, as amended (WSARA), requires the Secretary of Defense to modify guidance to ensure that the acquisition strategy for each major defense acquisition program provides for competitive prototypes before Milestone B approval--which authorizes entry into system development--unless the Milestone Decision Authority waives the requirement.[Footnote 1] Competitive prototyping, which involves commercial, government, or academic sources producing early prototypes of weapon systems or critical subsystems, can help Department of Defense (DOD) programs reduce technical risk, refine requirements, validate designs and cost estimates, and evaluate manufacturing processes prior to making major commitments of resources. It can also help reduce the time it takes to field a system, and as a result, reduce its acquisition cost. WSARA states that the Milestone Decision Authority may waive the competitive prototyping requirement only on the basis that (1) the cost of producing competitive prototypes exceeds the expected life-cycle benefits (in constant dollars) of producing such prototypes, including the benefits of improved performance and increased technological and design maturity that may be achieved through competitive prototyping; or (2) but for such a waiver, DOD would be unable to meet critical national security objectives. WSARA also provides that whenever a Milestone Decision Authority authorizes a waiver of the competitive prototyping requirement on the basis of what WSARA describes as "excessive cost," the Milestone Decision Authority is required to submit notification of the waiver, together with the rationale, to the Comptroller General of the United States at the same time it is submitted to the congressional defense committees. WSARA further provides that no later than 60 days after receipt of a notification of a waiver, we are mandated to review the rationale for the waiver and submit a written assessment of that rationale to the congressional defense committees.[Footnote 2] On December 3, 2013, we received notice from DOD that it had waived the competitive prototyping requirement for the Air Force's B-2 Defensive Management System Modernization (DMS-M) program. The Air Force's B-2 bomber is a low-observable, long-range strike aircraft capable of entering heavily defended areas to deliver both conventional and nuclear weapons. The defensive management system detects, identifies, and locates enemy radar systems and provides real- time threat avoidance, threat warning, and threat situational awareness information to the aircrew. The DMS-M program is expected to upgrade the aircraft's analog defensive management system to a digital capability, update pilot displays, enhance the in-flight capability to avoid unanticipated air defense threats, and improve frequency coverage and the reliability and maintainability of the system. The DMS-M consists of three critical subsystems--antenna, electronic support measures, and avionics and graphics processors--that will be integrated onto existing B-2 aircraft. The program is in the second of two technology development phases and plans to gain approval to enter system development in late fiscal year 2015. In this report, we assess DOD's rationale for waiving the competitive prototyping requirement for the B-2 DMS-M program and the analysis used to support it. To conduct our assessment, we compared the rationale in the waiver to the WSARA requirement to determine the extent to which the waiver is consistent with the statute. In addition, we reviewed the Air Force's cost-benefit analysis, which provides the data and assumptions on which the waiver is based, the technology development strategy, and other relevant documentation. We did not independently verify the Air Force's data on cost and benefits. We also submitted written questions to B-2 program officials to clarify information in program documentation, as necessary. We conducted this performance audit from March 2014 to April 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Results in Brief: DOD's rationale for waiving WSARA's competitive prototyping requirement for the B-2 DMS-M program addresses one of the two bases provided in the statute; namely that the cost of producing competitive prototypes exceeds the expected life-cycle benefits (in constant dollars) of producing the prototypes. The Air Force's B-2 DMS-M program is well into technology development and according to the cost- benefit analysis supporting the waiver, there is little opportunity to gain significant cost savings by prototyping. According to the Air Force, the prime contractor has already held competitions and selected suppliers that will leverage existing, technically mature subsystems for two of three DMS-M critical subsystems. The prime contractor has also conducted competitive prototyping for antennas, which are not yet technically mature, to reduce technical, cost, and schedule risk. In its cost-benefit analysis, the Air Force concluded that it will achieve significant operations and support cost reductions through its strategy of leveraging existing subsystems for the DMS-M and any additional improvements from prototyping would be on the margins. In the waiver, DOD concluded that the Air Force's prototyping cost- benefit analysis was reasonable. We also found that the Air Force's cost-benefit analysis was consistent with key principles in DOD's policy on economic analysis, which states that each feasible alternative for meeting an objective must be considered and its life- cycle costs and benefits evaluated.[Footnote 3] The Air Force analysis examined four different scenarios including the system-and subsystem- level prototyping options outlined in WSARA and DOD's implementing guidance. The Air Force concluded these strategies would increase the program's development costs by between $28.2 million and $524.8 million (in base year 2011 dollars) depending on the type and number of prototypes. The Air Force also estimated that the various prototyping strategies could achieve an estimated $1.3 million to $6.3 million (in base year 2011 dollars) in life-cycle benefits by improving the reliability of the system or key subsystems. Waiver Rationale Is Consistent with WSARA and the Supporting Analysis Considered a Reasonable Set of Prototyping Alternatives: DOD's rationale for waiving WSARA's competitive prototyping requirement for the B-2 DMS-M program addressed one of the two bases provided for a waiver in the statute, namely that the cost of producing competitive prototypes exceeds the expected life-cycle benefits, including the benefits of improved performance and increased technological and design maturity that may be achieved through competitive prototyping. The Air Force's B-2 DMS-M program is well into technology development and according to the cost benefit analysis supporting the waiver, there is little opportunity to gain significant cost savings by prototyping. During technology development, the prime contractor Northrop Grumman held competitions and selected suppliers for critical DMS-M subsystems. According to the Air Force, the chosen suppliers for the electronic support measures and avionics and graphics processors subsystems will leverage existing, technically mature subsystems and will also produce single prototypes of these subsystems during technology development. During technology development, the prime contractor also conducted competitive prototyping for B-2 DMS-M antennas, which are not yet technically mature, to reduce technical, cost, and schedule risk. According to the Air Force, there were four competitive prototypes produced for one part of the antenna subsystem and three competitive prototypes produced for the other part. In its cost-benefit analysis, the Air Force concluded that its strategy of leveraging existing subsystems for the DMS-M will achieve a 47 percent or $155.8 million (in base year 2011 dollars) reduction in operations and support costs over the current system and any additional improvements from prototyping would be on the margins. We also noted that the Air Force plans to use a cost-type contract for system development, which, according to DOD acquisition regulations, requires a written determination by the Milestone Decision Authority that a program is so complex and technically challenging that it would not be practicable to reduce program risk to a level that would permit the use of a fixed-price type contract.[Footnote 4] The Air Force recommended a cost-type contract because of uncertainty related to software development, system integration, testing, and potential funding volatility. In the waiver, DOD found reasonable the Air Force's cost-benefit analysis, which examined multiple prototyping strategies including system-and subsystem-level prototyping. The Air Force's cost-benefit approach was consistent with key principles in DOD's policy on economic analysis, which states that each feasible alternative for meeting an objective must be considered and its life-cycle costs and benefits evaluated.[Footnote 5] In all, the Air Force examined four different scenarios, including the system-and subsystem-level prototyping options outlined in WSARA and DOD's implementing guidance, and compared the life-cycle cost of each prototyping scenario to the potential life-cycle benefits. The Air Force concluded these strategies would increase the program's development costs by between $28.2 million to $524.8 million (in base year 2011 dollars) depending on the type and number of prototypes. The Air Force's prototyping cost estimates were based primarily on the actual and planned cost of the program's two technology development phases. For example, the Air Force assumed a second contractor would need to develop a DMS-M system prototype at the same cost as Northrop Grumman's system delivered during technology development. In addition, the system-level competitive prototyping scenario included $54 million for data rights because access to proprietary Northrop Grumman data is needed to act as the system integrator for B-2 related efforts. According to the Air Force, this estimate was developed using historic cost estimating factors. In practice, Northrop Grumman is unwilling to provide a price to the government for the data, so this prototyping scenario might not be feasible. The Air Force also estimated that the various prototyping strategies could achieve an estimated $1.3 million to $6.3 million (in base year 2011 dollars) in life-cycle benefits by improving the reliability of the system or key subsystems. According to the Air Force, it also considered the positive impacts of prototyping with respect to integration and production costs, but it did not include a dollar value for these benefits in any of its prototyping scenarios. Agency Comments and Our Evaluation: We provided a draft of this report to DOD for comment. In response, DOD stated that it had no comments. We are sending copies of this report to interested congressional committees, the Secretary of Defense and the Secretary of the Air Force. In addition, the report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report were Ron Schwenn, Assistant Director; Kristine Hassinger; Matthew B. Lea; Kenneth E. Patton; Carol D. Petersen; and Don M. Springman. Signed by: Michael J. Sullivan: Director, Acquisition and Sourcing Management: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James M. Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard J. Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Rodney Frelinghuysen: Chairman: The Honorable Pete Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Footnotes: [1] Pub. L. No. 111-23, § 203(a), as amended by the Ike Skelton National Defense Authorization Act for Fiscal Year 2011, Pub. L. No. 111-383, § 813. DOD modified its guidance related to the operation of its acquisition system through Directive-Type Memorandum (DTM) 09-027, "Implementation of Weapon Systems Acquisition Reform Act of 2009," (Dec. 4, 2009, incorporating Change 4, Jan. 11, 2013). This DTM also defined major defense acquisition programs as those estimated by DOD to require an eventual total expenditure for research, development, test, and evaluation or for procurement--including all planned increments--of more than $365 million or more than $2.19 billion, respectively, in fiscal year 2000 constant dollars. DOD issued an Interim Instruction 5000.02, "Operation of the Defense Acquisition System," on November 25, 2013 which canceled DTM 09-027 and modified the thresholds for major defense acquisition programs. However for purposes of this report we used DTM 09-027 because the competitive prototyping waiver for the B-2 Defensive Management System Modernization program was completed prior to issuance of the Interim DOD Instruction. The Milestone Decision Authority for major defense acquisition programs is the Under Secretary of Defense for Acquisition, Technology, and Logistics; the head of a DOD component; or, if delegated, the component acquisition executive. [2] Pub. L. No. 111-23, § 203(b)(1) & (2). [3] DOD Instruction 7041.3, Economic Analysis for Decisionmaking, Encl. 3, para. E3.1.1. (Nov. 7, 1995). [4] Defense Federal Acquisition Regulation Supplement § 234.004(2)(i)(C)(1). Cost-reimbursement contracts, also known as cost- type contracts, provide for payment of allowable incurred costs, to the extent prescribed in the contract. This contract type places most of the risk on the government, which may pay more than budgeted should incurred costs be more than expected when the contract is signed. [5] DOD Instruction 7041.3, Economic Analysis for Decisionmaking, Encl. 3, para. E3.1.1. (Nov. 7, 1995). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]