This is the accessible text file for GAO report number GAO-06-356 entitled 'Joint Strike Fighter: DOD Plans to Enter Production before Testing Demonstrates Acceptable Performance' which was released on March 15, 2006. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: March 2006: Joint Strike Fighter: DOD Plans to Enter Production before Testing Demonstrates Acceptable Performance: GAO-06-356: GAO Highlights: Highlights of GAO-06-356, a report to congressional committees: Why GAO Did This Study: The Joint Strike Fighter (JSF) is DOD’s most expensive aircraft program. The program represents 90 percent of the remaining planned investment for recapitalizing DOD’s aging tactical aircraft fleet. GAO is required by law to review the program annually for 5 years, beginning in fiscal year 2005. This is our second report and GAO assessed the program’s acquisition approach—in terms of capturing knowledge for key investment decisions—and identified an alternative to improve outcomes. What GAO Found: DOD is investing heavily in procuring JSF aircraft before flight testing proves it will perform as expected. For example, the JSF program plans to produce 424 low-rate initial production aircraft, at a total estimated cost of more than $49 billion, by 2013—the same time at which the program plans to complete initial operational testing. Producing aircraft before testing demonstrates the design is mature increases the likelihood of design changes that will lead to cost growth, schedule delays, and performance problems. Because the program will lack key design and testing knowledge, DOD plans to use cost reimbursement contracts to procure early production aircraft. This type of contract places a substantially greater cost risk on DOD and the taxpayers. Confidence that investment decisions will deliver expected capability within cost and schedule goals increases as testing proves the JSF will work as expected. Overlap of Production Investments and Testing: [See PDF for image] [End of figure] At the same time, the JSF program has not adopted an evolutionary approach to acquiring the aircraft—despite DOD policy that prefers such an approach. Instead, the JSF program has contracted to develop and deliver the aircraft’s full capability in a single-step, 12-year development program—a daunting task given the need to incorporate the technological advances that, according to DOD, represent a quantum leap in capability. DOD’s buying power has already been reduced. Since initial estimates, program acquisition unit costs have increased by 28 percent, or $23 million. Development costs have increased 84 percent, planned purchases have decreased by 535 aircraft, and the completion of development has slipped 5 years, delaying delivery of capabilities to the warfighter. With more than 90 percent of the JSF investment remaining, DOD officials have the opportunity to adopt a knowledge- based and evolutionary acquisition strategy that would maximize DOD’s return on its investment. The acquisition approach used for the F-16 fighter, the Air Force’s JSF predecessor, could provide a model for delivering JSF capabilities to the warfighter sooner and recapitalizing tactical aircraft forces more quickly while lowering risk. The F-16 program successfully evolved capabilities over the span of about 30 years, with an initial capability delivered to the warfighter about 4 years after development started. What GAO Recommends: The Congress should consider delaying authorizations and appropriations for JSF procurement until a new business case is developed and flight testing demonstrates the design and integrated mission systems work. GAO included this matter for consideration because DOD did not plan to make changes as a result of recommendations. GAO is recommending that DOD delay investing in production until flight testing shows that the JSF performs as expected, and that the program develop a plan, consistent with DOD’s preferred policy, to adopt an evolutionary approach that limits new content for each increment to proven technologies and design. DOD partially concurred, but believes that its current practices achieve our recommendations’ objectives. www.gao.gov/cgi-bin/getrpt?GAO-06-356. To view the full product, including the scope and methodology, click on the link above. For more information, contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: JSF Acquisition Strategy Will Begin Procurement before Testing Demonstrates the Aircraft's Performance: Evolutionary Acquisition Strategy Provides a Less Risky Alternative for the JSF Program: Conclusions: Matters for Congressional Consideration: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Changes in JSF Program Purchase Quantities and Costs: Appendix III: Comments from the Department of Defense: Table: Table 1: Military Services' Planned Use for the Joint Strike Fighter: Figures: Figure 1: Overlap of Low-Rate Initial Production Investments and Testing Demonstrations of the JSF Variants: Figure 2: JSF Program Acquisition: Figure 3: JSF's Annual Funding Requirements, as of December 2004: Figure 4: F-16 Incremental Development Approach: Figure 5: Planned JSF Blocks: Figure 6: Potential Application of an Incremental Development Approach to the JSF Program: Abbreviations: DOD: Department of Defense: JSF: Joint Strike Fighter: OSD: Office of the Secretary of Defense: United States Government Accountability Office: Washington, DC 20548: March 15, 2006: Congressional Committees: Since the Joint Strike Fighter (JSF) program began in 1996, Congress has appropriated nearly $25 billion for its development. It is the Department of Defense's (DOD) most expensive aircraft acquisition program. As the program is currently planned, DOD estimates it will spend $257 billion to develop and procure about 2,443 aircraft and related support equipment by 2027 and an additional $347 billion to operate and support these aircraft once they have been fielded. JSF program goals are to develop and field a family of stealthy, strike fighter aircraft for the Navy, Air Force, and Marine Corps, and United States allies. Design plans call for three variants of the aircraft, but the program aims to provide maximum commonality to minimize life cycle costs. JSF is a central part of DOD's overall recapitalization strategy for its tactical aircraft fleet, representing 90 percent of the remaining planned investment for its major tactical aircraft programs. How DOD manages the JSF program in the future will be critical for getting the most out of this remaining large investment and for success in replacing the aging fleet of tactical aircraft. The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (P.L. 108-375) requires GAO to review the JSF program annually for 5 years.[Footnote 1] In March 2005, we reported the JSF's original business case was unexecutable and recommended that DOD establish an executable program consistent with best practices and DOD policy regarding evolutionary acquisitions.[Footnote 2] We also recommended that this new business case be accompanied by a knowledge-based acquisition approach[Footnote 3]--an approach that ensures attainment and use of demonstrated product knowledge before making future investments. In commenting on our report, DOD stated that JSF's restructured acquisition plan would incorporate a knowledge-based, evolutionary approach consistent with DOD policy. This is our second report. Specifically, we (1) determined whether the JSF program acquisition strategy captures critical knowledge in time to make production investment decisions and (2) identified an alternative to the current acquisition strategy to improve JSF program outcomes. The act also requires us to certify whether we had access to sufficient information to make informed judgments on the matters contained in our report. We were provided sufficient information to assess the plans to capture technology, design, and manufacturing knowledge for the three JSF variants, the evolutionary nature of the acquisition strategy, and the opportunities to improve outcomes. At the time of our review, the Office of the Secretary of Defense (OSD) was still preparing its fiscal year 2007 budget request as well as conducting its independent assessment of the program cost estimate, which is not expected to be completed until sometime in 2006. Additionally, the Quadrennial Defense Review was ongoing during our review, and we did not have insights into the potential outcomes or how the results of the study might affect the JSF program. We performed our work from June 2005 through March 2006 in accordance with generally accepted government auditing standards. For more on our scope and methodology, see appendix I. Results in Brief: Despite recent program restructuring, the JSF program continues to base its acquisition strategy on a highly concurrent approach that makes significant investments in manufacturing capabilities and production aircraft before flight testing demonstrates the JSF's performance. As a result, significant development risk remains, and it is likely that current cost and schedule goals will not be met. The program plans to proceed into low-rate initial production in 2007 with inadequate testing to prove a mature design for any of the three basic JSF variant airframes, without developing critical software, or without a fully integrated aircraft with advanced mission systems and prognostic maintenance capabilities. All three production representative variants[Footnote 4] will not be in flight testing until 2009, and a fully configured, integrated development aircraft will not begin flight testing until 2011--4 years after production begins. By this time, DOD plans to have ordered 190 aircraft at a cost of about $26 billion. By 2013, when initial operational testing is expected to be complete, the program plans to have procured 424 aircraft at an expected cost of $49 billion. Because it will lack necessary technology, design, and performance knowledge, DOD plans to procure early production aircraft using cost-reimbursable-type contracts. This type of contract places a significantly greater cost risk on DOD. DOD has revised its acquisition policy to embrace evolutionary acquisition, allowing managers to develop and evolve a product through small, time-phased development increments--an alternative still available to the JSF program. Instead of establishing time-phased requirements for aircraft to be delivered in sequence that could first meet DOD's need to recapitalize its aging fleet of aircraft and then evolve the aircraft to eventually achieve improved capabilities in future increments, DOD chose a single-step approach to develop and deliver the JSF with ultimate capabilities in a single 12-year system development program. The large amount of uncertainty in this approach has already led to poor cost and schedule estimates for the JSF program and a reduction in DOD's buying power. The length and scope of work remaining continue to make it difficult to predict the cost and time needed to complete the program. The Air Force's JSF predecessor, the F- 16 Fighter program, provides a model for a less risky alternative JSF acquisition strategy that delivers weapons to the warfighter more quickly and that recapitalizes tactical aircraft forces sooner. The F- 16 program successfully delivered 2,200 aircraft with incremental improvements as technology became available over the span of about 30 years. An initial F-16 capability was delivered to the warfighter within about 4 years after development began and substantial quantities were purchased with each increment to meet warfighter needs, including the replacement of aging legacy tactical fighter aircraft. To improve the chances for a successful outcome, we are recommending the JSF program delay production and investments in production capability until the aircraft design qualities and integrated mission capabilities of the fully configured and integrated JSF aircraft variants have been proven to work in flight testing. We are also recommending the Secretary of Defense report to the Congress by July 2006 on the feasibility of an incremental acquisition approach that follows the intent of DOD evolutionary acquisition policy and delivers a first increment that limits new content to proven technologies and design. DOD partially concurred with our recommendations stating that the current JSF acquisition strategy and management practices will be able to achieve the objectives of our recommendations, and therefore, further limits on production are unnecessary. DOD also stated that the program's strategy balances technical risk, financial resources, and operational needs to reduce program risk and is based on a knowledge- based, incremental approach. We continue to believe DOD's planned strategy to make significant investments in production concurrent with testing is very risky and is similar to strategies of past programs with poor outcomes. We believe that DOD has an opportunity to reduce risk by adopting an acquisition strategy based on capturing technology, design, and manufacturing knowledge. We also believe that smaller, more manageable commitments in capabilities would make JSF program cost and schedule more predictable and deliver needed capabilities to the warfighter sooner. Because DOD does not plan to make changes as a result of our recommendations, we believe Congress should consider delaying authorizations and appropriations for JSF procurement until DOD develops a new business case and demonstrates the aircraft design qualities and integrated mission capabilities of the fully configured and integrated JSF variants work as designed based on actual flight testing. Background: JSF is a joint, multinational acquisition program for the Air Force, Navy, Marine Corps, and eight cooperative international partners. The program began in November 1996 with a 5-year competition between Lockheed Martin and Boeing to determine the most capable and affordable preliminary aircraft design. Lockheed Martin won the competition, and the program entered system development and demonstration in October 2001. DOD has set major expectations for the program. The program's objective is to develop and deploy a technically superior and affordable fleet of aircraft that support the warfighter in performing a wide range of missions in a variety of theaters. The single-seat, single-engine aircraft is being designed to be self-sufficient or part of a multisystem and multiservice operation, and to rapidly transition between air-to-surface and air-to-air missions while still airborne. To achieve its mission, JSF will incorporate low observable technologies, defensive avionics, advanced onboard and offboard sensor fusion,[Footnote 5] internal and external weapons, and advanced prognostic maintenance capability. According to DOD, these technologies represent a quantum leap over legacy tactical aircraft capabilities. At the same time, the JSF aircraft design includes three variants: a conventional takeoff and landing variant for the Air Force; an aircraft carrier-suitable variant for the Navy; and a short takeoff and vertical landing variant for the Marine Corps, the Air Force, and the United Kingdom. JSF is intended to replace a substantial number of aging fighter and attack aircraft in DOD's current inventory (see table 1). Table 1: Military Services' Planned Use for the Joint Strike Fighter: Service: Air Force; Planned use: Replacement for the F-16 and A-10; complement the F-22A. Service: Marine Corps; Planned use: Replacement for the AV-8B and F/A- 18 A/C/D. Service: Navy; Planned use: Complement the F/A-18 E/F. Source: DOD data. [End of table] In recent years, DOD has revised its acquisition policy to embrace an evolutionary, or incremental, approach to improve program outcomes. The acquisition policy states evolutionary acquisition is the preferred DOD strategy for rapid acquisition of mature technology for the user by delivering capability in increments, recognizing up front the need for future capability improvements. The objective is to balance needs with resources in order to put capability into the hands of the warfighter more quickly. The policy states that a product enters system development and demonstration when an affordable increment of militarily useful capability has been identified, technology has been demonstrated in a relevant environment, and a system can be developed for production in a short time frame (normally less than 5 years). Each increment of an evolutionary acquisition will have its own decision milestones and baseline--cost, schedule, and performance requirements. In commenting on our March 2005 JSF report, DOD stated that JSF's restructured acquisition plan would balance technical, cost, and schedule risk and incorporate a knowledge-based, evolutionary approach to system acquisition consistent with DOD policy. JSF Acquisition Strategy Will Begin Procurement before Testing Demonstrates the Aircraft's Performance: DOD's acquisition strategy for JSF has not substantially changed as a result of the program's restructuring last year. In May 2005, DOD leadership approved the program's plan, which intends to start production in 2007--more than 2 years before all three JSF variants have completed some flight testing of the aircraft's basic design, 4 years before a fully configured and integrated aircraft is expected to be flight tested, and 6 years before development and initial operational testing are scheduled to be completed.[Footnote 6] Before development has ended, DOD will have potentially signed procurement contracts for 424 JSF aircraft at an estimated cost of $49 billion. Starting production before ensuring the design is mature through flight testing significantly increases the risk of costly design changes that will push the program over budget and behind schedule. Evidence of the risk associated with concurrently developing, testing, and producing the JSF aircraft can be seen in the program office strategy to place initial production orders on a cost reimbursement contract, placing an unusually high risk burden on the government during the early production phase. Key Testing Events Will Not Be Completed before Significant Procurement Investments Are Made: The JSF program plans to begin low-rate initial production of the aircraft before many of JSF's technology advances and basic flying qualities are flight-tested and to produce substantial quantities of aircraft before initial operational testing is completed. According to JSF program plans, DOD's low-rate initial production quantities will increase from 5 aircraft a year in 2007 to 133 a year, before development and initial operational testing are completed in 2013.[Footnote 7] This production rate will require DOD to invest significantly in tooling, facilities, and personnel. Initial contractor cost estimates indicate that close to $1 billion of new tooling will be needed to support low-rate initial production rates. Early contractor data also show that the program will need to double its manufacturing workforce by 2008 and will need six times the number of manufacturing personnel before low-rate initial production ends. Total monthly spending for production activities in 2013 is expected to approach $1 billion, a significant increase from $100 million a month when production is scheduled to begin in 2007. The cost of discovering design problems during production could be significant if testing shows that large, structural components of the aircraft require modifications. Design changes needed in one variant could also ripple through the other two variants, reducing efficiencies necessary to lower production and operational costs with common parts and manufacturing processes for the three variants. Some industry officials have indicated that the cost of design changes such as these could be 10 to 1,000 times greater, depending on how far the product has progressed into production. When the JSF program is expected to begin low-rate initial production in 2007,[Footnote 8] the program will have completed less than 1 percent of the flight test program and none of the three JSF variants will have a production representative prototype built and in flight testing. Features critical to JSF's operational success, such as the low-observable and highly common airframe and the advanced mission systems, will not have been demonstrated in a flight test environment. The program plans to proceed into low-rate initial production without demonstrating that (1) the aircraft's flying qualities function within the parameters of the flight envelope--that is, the set limits for altitude, speed, and angles of attack; (2) the aircraft design is reliable; or (3) a fully integrated and capable aircraft system can perform as intended. These are key indicators of a program's readiness for entering production and making significant investments in tooling, facilities, and materials. When the first fully integrated and capable development JSF is expected to fly, in 2011, DOD will have committed to buy 190 aircraft at an estimated cost of $26 billion under current plans. By the time testing is planned to be completed, in 2013, DOD will have procured more than double that amount--424 aircraft at an estimated cost of about $49 billion. Figure 1 shows the significant overlap in development and testing and the major investments in production capability and aircraft that inject additional cost and delivery risks that could delay delivery of a proven capability to the warfighter. Figure 1: Overlap of Low-Rate Initial Production Investments and Testing Demonstrations of the JSF Variants: [See PDF for image] [A] Flight testing data reflect the percentage of the total flight tests completed at the time of the planned investment decision, which is currently planned for January of each year. [End of figure] Under the current schedule, the JSF program plans to manufacture and deliver 15 flight test aircraft and 7 ground test (nonflying) articles in 5 years--an aggressive schedule when compared with schedules of other programs with fewer variables. Current JSF schedules are already showing delivery of early test aircraft could be later than the planned delivery date. Unplanned manufacturing and technical problems can delay the completion of a flight test program, increase the number of flight test hours needed to verify that the system will work as intended, and affect when key knowledge will be available for making investment decisions. For example, when the B-2 program began flight testing in July 1989, it estimated that the flight test program would last approximately 4.5 years and require about 3,600 flight test hours. When the test program ended in 1997, the flight test hours had grown to 5,000 hours, or by 40 percent, over an 8-year period. Program officials cited several causes, including difficulties in manufacturing test aircraft and correcting deficiencies from unanticipated problems discovered during testing. The F-22A encountered similar delays, increasing a planned 4-year flight test program to about 8 years and affecting the program's ability to conduct operational testing and move into production on schedule. Initial Procurement Decisions Will Be Made before the Basic Aircraft Design Is Demonstrated: While each JSF variant is similar--all are being designed to have stealth airframes, fly at supersonic speeds, shoot air-to-air missiles, and drop bombs on a target--there are subtle airframe design differences to support the services' different operational concepts and environments. Test officials acknowledge that each airframe variant will require flight testing to demonstrate that each will fly as intended. Yet at the time the JSF program expects to begin low-rate initial production, only 1 of 15 flight test aircraft is expected to have started testing. Further, the aircraft's test flights are not scheduled to begin until August 2006, less than 6 months before the planned decision to begin low-rate initial production,[Footnote 9] and will not assess the aircraft in more demanding operational environments, such as high angles of attack. Moreover, the first flight test aircraft will not include a large number of design changes that resulted from an effort to significantly reduce aircraft weight over the past 2 years and, therefore, will not represent the planned production configuration. The first demonstration of a production representative airframe that includes the latest design changes is scheduled for late 2007--after production has been initiated--with first flight of the short takeoff and vertical landing variant. All three variants will not be in flight testing until 2009, when the carrier variant is expected to have its first flight. Several key test events identified by the contractor to demonstrate the flight characteristics of these aircraft in the intended operational environment are not scheduled until 2009 or later. These include shipboard operations for the carrier and short takeoff and vertical landing variants. According to the contractor, the full-flight envelope for all three variants will not be fully demonstrated until 2011 or 2012--4 to 5 years after low-rate initial production is scheduled to begin. According to DOD, modeling and simulation will be used to evaluate the performance of the aircraft's flying qualities and to support decisions to invest in production. However, the Rand Corporation recently reported in a study on testing and evaluation that modeling is not a substitute for flight testing.[Footnote 10] Rand found that even in performance areas that are well understood, it is not unusual for flight testing to uncover problems that were not apparent in simulations. Examples include flight effects on the wing of the F/A-18 EF and buffeting of stores externally carried on various aircraft when flown in certain conditions. Additionally, OSD testing officials indicated that flight testing of each variant is necessary to demonstrate designed capabilities. Our past work has found that flying quality problems were identified during actual flight testing on programs like the F-22A, B-2A, and V-22. Finally, two major ground tests of the airframe's structural integrity- -fatigue and static testing--will be in their very early stages or not have started when production begins. Fatigue testing, which measures the aircraft's durability over its expected life, is slated to begin in 2008 with testing to show structural fatigue over one lifetime of flying to be completed in 2009 and a second lifetime completed by 2010. Fatigue testing uses actual JSF airframe structures that are subjected to the long-term stresses expected over the aircraft's life. Standards require this testing to be done to represent the stresses expected over two lifetimes. Static testing, which measures the aircraft's ability to withstand the stresses expected to be encountered throughout the aircraft's flight regime, is slated to begin in 2007 and last through 2009. These tests are important to reduce the risk of structural problems emerging during production or after aircraft are fielded. Mission Capabilities Will Not Be Tested until 4 Years after Procurement Has Begun: In addition to late design testing, many of the mission systems planned for JSF will not be available for initial flight testing either. Although laboratory tests are under way, DOD does not plan to flight- test several of the new technologies needed for the JSF to perform its intended missions until 2009 at the earliest. Defense Operational Test and Evaluation officials have stated that flight-testing capabilities in a production representative test aircraft and in the operational environment planned for the new system are important to reducing risk. This actual environment differs from what can be demonstrated in a laboratory and has historically identified unexpected problems. For example, the F-22A software worked as expected in the laboratory, but when tested in the aircraft, significant problems were identified. These problems delayed testing and the delivery of a proven capability. The different levels of mission capability will be tested in JSF aircraft as follows. * Block 1, an initial air interdiction capability, is scheduled to begin testing in 2009, with initial operational testing scheduled for 2011--4 years after DOD plans to begin production. * Block 2, an improved air interdiction and close air support capability, is scheduled to begin testing in 2010, with initial operational testing scheduled for 2012. This block will include several critical technologies that are not fully mature, such as the advanced missions systems and prognostics and health maintenance, but are critical to meeting requirements like sortie generation and mission capabilities. * Block 3, the fully integrated and capable JSF, is scheduled to begin testing in 2011. At this time less than 50 percent of the planned mission capability testing will have been completed. This is close to the same point that the F-22A and other past programs experienced difficulties integrating all the complex software and hardware components into the aircraft. Flight testing to evaluate the effectiveness and suitability of the fully integrated system is expected to continue until the full-rate decision in late 2013. Aggressive Software Development Schedule May Cause Delays: JSF's expected performance is largely dependent on demonstrating software that supports vehicle, mission system, and other capabilities. The program plans to develop over 19 million lines of code-- substantially more than the lines of code needed for the F-22A. The software is planned to be developed in five blocks. The first block is scheduled for completion in 2006, and the last block is scheduled for completion in 2011. At the time the program enters low-rate initial production, the program will have completed less than 35 percent of the software needed for the system's functionality. Past programs have encountered difficulties in developing software, delaying flight test schedules. Data provided by the program office indicate that the program is already showing early signs of falling behind its software delivery schedule. JSF program officials recognize the risk associated with this large software effort. The Current Acquisition Strategy Requires Prolonged Reliance on Cost Reimbursement Contracts That Place an Increased Risk on DOD: According to program officials, the uncertainties inherent in concurrently developing, testing, and producing the JSF aircraft prevent the pricing of initial production orders on a fixed price basis. Consequently, the program office plans to place initial production orders on a cost reimbursement basis. Cost reimbursement contracts provide for payment of allowable incurred costs, to the extent prescribed in the contract. Such contracts are generally used only when the uncertainties involved in contract performance do not permit costs to be estimated with sufficient accuracy to use any type of fixed price contract. Cost reimbursement contracts place substantial risk on the buyer--in this case DOD--because the contractor's responsibility for performance costs is minimized or reduced. In contrast, a fixed price contract provides for a pre-established price, and places more risk and responsibility for costs and resulting profit or loss on the contractor and provides more incentive for efficient and economical performance. The program plans to transition to fixed-price-type contracts once the air vehicle has a mature design, has been demonstrated in flight test, and is producible at established cost targets. According to program officials, this transition will occur sometime before full-rate production begins in 2013. The program office believes the combination of the early concept development work, the block development approach, and the relatively small numbers of aircraft in the initial production buys allow decisions to be made earlier than normal with an acceptable level of risk. Evolutionary Acquisition Strategy Provides a Less Risky Alternative for the JSF Program: The JSF acquisition strategy currently plans a single-step approach to deliver a quantum leap in tactical fighter capability by 2013 and has already felt the negative cost and schedule impacts from the executing this approach. The length and scope of the remaining effort in the JSF program make it even more difficult to accurately estimate cost and delivery schedules. The JSF funding profile--which requires an average of $11 billion annually for the next two decades--is also at risk to increase if costs continue to grow or schedules are further delayed to develop the ultimate JSF capabilities. An alternative acquisition strategy, such as used by the F-16 program, that sequences capabilities over time based on proven technologies and design would reduce risk and deliver aircraft sooner. This evolutionary approach is actually the preferred approach in DOD's acquisition policy for acquiring new systems for more rapid delivery of incremental capabilities to the warfighter. JSF Program Is Designed to Deliver Full Required Capability in a Single- Step Development Program: Instead of establishing time-phased requirements for aircraft to be delivered in sequence that could first meet DOD's need to recapitalize its aging fleet of aircraft and then evolve the aircraft to eventually achieve improved capabilities in future system development increments, DOD chose a single-step development approach to deliver the full required capability by the end of system development in 2013. That approach is now planned to last 17 years (5 years in concept development and 12 years in system development)--including the development of immature, undemonstrated technologies and will take at least two to three times longer than the development time of leading commercial firms or that suggested by defense acquisition policy. The JSF acquisition strategy is shown in figure 2. Figure 2: JSF Program Acquisition: [See PDF for image] [A] The bar for production is not drawn to scale. JSF production lasts for 20 years. [End of figure] While JSF's acquisition strategy calls for delivering a small number of aircraft with limited capabilities, the program has committed to deliver the full required capability by the end of system development and demonstration in 2013. For JSF, this includes a set of objectives that exceeds those of aircraft development programs of the past. JSF will use cutting-edge technology to fuse data from the aircraft itself and other sources, from all aspects of the aircraft, and display the information to the pilot. The aircraft must be able to rapidly transition from ground attack to air-to-air missions while airborne. JSF also expects advances in technologies for mission systems, prognostics, and autonomic logistics support requiring hardware development as well as extensive and complex software development and integration to have lower cost of ownership than the legacy aircraft while being able to deploy rapidly, sustain high mission reliability, and maintain a high sortie generation rate. Past single-step development programs have been unsuccessful in predicting acquisition costs and delivery schedules. For example, development costs for the F-22A fighter, B-2 bomber, Crusader artillery vehicle, and Comanche helicopter skyrocketed, and production quantities were either substantially reduced or the program was canceled. Such outcomes are a poor return on investment and a failure to recapitalize aging legacy systems. So far the JSF program is experiencing similar results. Since the program's start, development cost has increased 84 percent, the development schedule has increased by almost 5 years, and planned delivery of capabilities to the warfighter has been delayed. DOD now plans to buy 535 fewer aircraft than originally planned. As a result, DOD's buying power has been significantly reduced--program acquisition unit costs have increased by 28 percent, or $23 million, since first estimates. See appendix II for more details on JSF outcomes. Funding Needs Could Increase Given the Level of Risk to Complete the JSF Program: Despite a lengthy restructuring effort that added over $7 billion and 18 months to the development program, JSF officials have stated that the restructured program has little or no flexibility for future changes or unanticipated risks. Furthermore, the length and scope of the remaining effort make it even more difficult to accurately estimate cost and completion schedules. While it has been over 9 years since the program started, the first flight test aircraft still has not been delivered. The program has planned almost 8 years to complete the remaining activities of the system development and demonstration phase. These remaining activities include: * fully maturing 7 of the 8 critical technologies; * completing the designs and releasing the engineering drawings for all three variants; * manufacturing and delivering 15 flight test aircraft and 7 ground test articles; * developing 19 million lines of software code; and: * completing a 7-year, over 12,000-hour flight test program. The JSF program's latest planned funding profile for development and procurement, produced in December 2004, expects annual funding requirements to hover close to $13 billion in between 2012 and 2022, peaking at $13.8 billion in 2013. If the program fails to achieve its current estimated costs, funding challenges could be even greater than they are today. Thus, even a modest cost increase would have dramatic impacts on funding. For example, a 10 percent increase in production costs would amount to over $21 billion (see fig. 3). Figure 3: JSF's Annual Funding Requirements, as of December 2004: [See PDF for image] [End of figure] The current cost estimate reflects the position of the JSF program office. The Office of Secretary of Defense Cost Analysis Improvement Group was to update its formal independent cost estimate in the spring of 2005, and the Navy and the Air Force were expected to fully fund the program consistent with the estimate. The group now does not expect to formally complete its estimate until the spring of 2006, but its preliminary estimate was substantially higher than the program office's. According to Cost Analysis Improvement Group officials, an assessment of the software and mission systems requirements based on more recent information could further increase the estimate of JSF costs. An Incremental Development Approach Provides an Alternative Model for Reducing JSF Risks: An incremental development approach consistent with DOD's policy on evolutionary acquisition and best practices has potential to deliver warfighter capabilities in planned product increments that would increase JSF capabilities over time. The F-16 fighter program, the Air Force's JSF predecessor, successfully evolved capabilities over the span of about 30 years, delivering increases of capabilities quickly and often, as technologies became available. That program may provide a model for a possible alternative acquisition strategy for the JSF program. Structuring the program into separate and manageable increments based on what is achievable now and in the future would allow more predictable cost and delivery estimates. Over the past three decades, the Air Force successfully procured more than 2,200 F-16s. The F-16 acquisition approach allowed the timely and affordable delivery of aircraft and capability to meet the warfighter's needs, including the recapitalization of aging aircraft. By using an evolutionary approach to develop the aircraft, the program was able to quickly deliver new and improved capabilities to the warfighter and increase the aircraft's capability as new technologies were matured and added to the aircraft. The first increment, developed during the 1970s, provided a "day fighter" aircraft with basic air-to-air and air-to- ground capabilities. This allowed the developer to deliver new and useful military capability to the warfighter in less than 4 years. With each subsequent increment, new technology was used to improve the engine, radar, structure, avionics, and other systems that allow the aircraft today to perform close air support, ground attack, air defense, and suppression of enemy defense missions (see fig. 4). The evolutionary approach also enriched the industrial base capabilities by extending the life of the production over the length of this incremental approach. Figure 4: F-16 Incremental Development Approach: [See PDF for image] Note: This gives the number of aircraft delivered to the U.S. Air Force only, no foreign military sales or other allied government sales included. [End of figure] In contrast, JSF's fully configured design represents a quantum leap in capability that far exceeds the capability of legacy systems that JSF is intended to replace. While the program is using a block structure-- where each block adds capabilities over the proceeding block--the blocks are part of a single development effort, and DOD is on contract with the developer to deliver the warfighter the full capability (see fig. 5). The program's block structure provides for an escalating capability, but DOD already plans to buy 95 percent of JSF aircraft with the ultimate capability (block 3). Unlike the approach used with the F-16, this risky approach will likely be like past programs that have encountered significant increases in cost and time, not allowing DOD to quickly recapitalize the aging legacy aircraft. Figure 5: Planned JSF Blocks: [See PDF for image] [End of figure] An evolutionary approach would entail delivering a first product increment with at least as much capability as currently operating legacy aircraft with sufficient quantities to allow DOD to retire older planes sooner and reduce operating inefficiencies. DOD has repeatedly raised concerns about the age of its fighter aircraft fleet, which was bought largely in the 1970s and 1980s and will need to be replaced around 2010. Delays in fielding JSF aircraft may increase costs to maintain legacy aircraft to meet force structure requirements. Limiting development to 5-year increments or less, as suggested in the DOD acquisition policy, would force smaller, more manageable commitments in capabilities and make costs and schedules more predictable. Some of the more challenging JSF capabilities, such as the advanced mission systems or prognostics technologies, would be deferred, kept in the technology base (off the critical path of the JSF program), and added to follow-on increments once they are demonstrated in the more conducive technology development environment.[Footnote 11] In addition to considering available resources--including technology and design knowledge, budget, and time--each increment should be based on the warfighter's most immediate needs and the number of aircraft needed to maintain a viable fleet. Figure 6 shows what an incremental approach might look like for JSF. Figure 6: Potential Application of an Incremental Development Approach to the JSF Program: [See PDF for image] [A] The bar for production is not drawn to scale. JSF production lasts for 20 years. [End of figure] Because an incremental approach would reduce the amount of risk in each development phase, it would make the program manager and contractor accountable for a lower-risk system development phase with precise cost and schedule targets. A shorter system development phase also makes it more feasible to align a program manager's tenure to the completion of the phase, holding the program manager accountable for decisions made. It also would enable the use of fixed-price-type contracts for production that contains a pricing structure that reduces government risk. Conclusions: While the JSF program plays a critical role in DOD plans to recapitalize the services' aging tactical aircraft fleet, DOD's current acquisition approach for the JSF program continues to be risky and could further jeopardize meeting this important objective. The JSF program has already encountered increases to estimated development costs, delays to planned deliveries, and reductions in the planned number of JSF to be procured that have eroded DOD's buying power. We believe the current acquisition strategy to begin production in 2007 is too risky. By committing to procure large quantities of the aircraft before completing testing that demonstrates that the design is mature and reliable, DOD has significantly increased the risk of further compromising its return on investment--as well as adding more cost and delaying the delivery of critical capabilities to the warfighter. Also, making sizable investments in tooling and other manufacturing capability needed to produce JSFs at higher rates before a fully integrated aircraft is demonstrated in testing places the program at risk for expensive design changes as testing uncovers problems. Deferring production decisions until all three variants' performance has been demonstrated throughout their flight envelopes and with the full integrated capability would allow additional time to capture knowledge and help to significantly minimize these risks. To continue with the current plan to deliver the ultimate JSF capability will require (1) execution of a 7-year test program without further schedule delays, (2) development of 5 major blocks of software (19 million lines of code), and (3) new and yet undemonstrated technologies for advanced mission systems that must collect, analyze, and synthesize information from other platforms or sensors not all a part of the JSF aircraft. These are necessary to ensure the JSF has increased situational awareness to destroy enemy air defenses and perform deep strike missions with advanced weapons. The JSF program will also need, annually, an average of $11 billion of development and procurement funding for the next two decades. All of these factors add to the challenges faced by DOD in trying to manage this highly concurrent program, increasing the risk it will have the same poor outcomes experienced by similar major acquisition programs with significant cost and schedule growth and delayed modernization and recapitalization of aging systems. DOD has an alternative. With 90 percent of DOD's remaining planned investment in JSF, it can adopt a knowledge-driven and evolutionary acquisition approach to reduce JSF program risks, recapitalize its aging tactical air force sooner, and deliver needed capabilities to the warfighter more quickly. The experiences of the F-16 program--evolving and improving capabilities over time while providing the warfighter combat capability--provide a precedent for this. Requirements for the first increment of JSF under this evolutionary approach would match a level supported by current knowledge of technologies and design. This would allow the testing of those reduced requirements to support a knowledge- based low-rate production decision sooner and allow delivery of a useful product and in sufficient quantities to start replacing the aging legacy fighter force. Capabilities that demand as yet undemonstrated technologies, for example, advanced mission systems and software, would be included as requirements in subsequent future JSF aircraft increments--managed as separate development programs--as technology is demonstrated in the technology base and becomes available. Matters for Congressional Consideration: DOD does not plan to change its business case or acquisition plan for developing and buying the JSF. Without changes, the acquisition plan will put at risk $50 billion for procuring JSF aircraft at the same time the program develops and tests the aircraft's expected performance capabilities over a 7-year, 12,000 hour flight test program. The JSF program has continually missed its cost and schedule targets over the 5 years it has been in development. If DOD were to make smaller, more manageable commitments in capabilities, it would make cost and schedule more predictable and deliver needed capabilities to the war fighter sooner. For these reasons, Congress may want to consider delaying authorizations and appropriations for procuring JSF aircraft until: 1. DOD develops a knowledge-based business case that matches requirements with proven technologies and design knowledge and available funding. Capabilities that demand technological advances not yet demonstrated should be part of future increments that are funded and managed separately once demonstrated. 2. DOD demonstrates the aircraft design qualities and integrated mission capabilities of the fully configured and integrated JSF variants work as designed based on actual flight testing. Recommendations for Executive Action: We are making the following recommendations to reduce program risk and increase the likelihood of successful program outcomes by delivering capabilities to the warfighter when needed and within available resources. Specifically, we recommend that the Secretary of Defense: 3. Delay the investment in production aircraft until sufficient testing has at least demonstrated the basic airframe design of each JSF variant in important parts of the flight envelope, including combinations of critical altitudes, speeds, and angles of attack. 4. Once the basic airframe design has been demonstrated, limit production quantities to the number of aircraft needed for operational testing and that can be produced using the current production capability (based on existing tooling, materials, and personnel) until a fully integrated aircraft (with advanced mission systems and predictive maintenance systems) demonstrates through flight testing the required capabilities. 5. Plan an incremental acquisition strategy that follows the intent of DOD evolutionary acquisition policy and delivers a first increment that limits new content to proven technologies and design. The plan should be completed and reported to the Congress by July 2006, and include adjustments in associated programs as necessary to maintain needed capabilities. Agency Comments and Our Evaluation: DOD provided us with written comments on a draft of this report. The comments appear in appendix III. DOD partially concurred with our recommendation that the Secretary of Defense delay the investment in production aircraft until sufficient testing has at least demonstrated the basic airframe design of each JSF variant in important parts of the flight envelope, including combinations of critical altitudes, speeds, and angles of attack. DOD agreed that a knowledge-based approach is critical to making prudent decisions and that this type of approach is being used by JSF. However, DOD's response did not provide a plan of action to show how it will ensure flight testing demonstrates acceptable performance before significant production investments are made. In the past, Congress has raised concerns about the costly outcomes of highly concurrent development and production efforts that are not "flying before buying." DOD points out that the JSF program is in its fifth year of a 12-year development, but starting production with nearly 7 years of development and testing remaining leaves a high probability that, because it is not flying before buying, it will incur costly design changes and delay getting capability to the warfighter. Until there has been testing to demonstrate that all three variants will work as intended, we believe entering production in 2007 places financial investments at significant risk. The risk increases as spending for production goes from about $100 million a month in 2007 to over $500 million a month just 2 years later. DOD partially concurred with our recommendation to the Secretary of Defense to limit production quantities to the number of aircraft needed for operational testing and that can be produced using the current production capability until a fully integrated aircraft demonstrates the required capabilities through flight testing. DOD stated it is limiting production until aircraft capabilities are fully demonstrated and that further limits are not necessary. It stated that various program reviews to assess performance, including test objectives, are conducted to ensure associated program risks are understood and mitigated. We believe DOD's plan to invest almost $50 billion to buy over 400 aircraft concurrent with testing is very risky. Significant efforts remain in the JSF program to demonstrate the aircraft will perform as expected. A number of major DOD acquisition programs have employed highly concurrent acquisition strategies in the past and experienced significant cost increases and schedule delays. DOD stated its use of modeling and simulation and laboratory testing reduces risk. However, DOD operational test officials have stated that demonstrations need to occur in the actual aircraft in an operational environment to verify that the system works as intended. For example, the F-22A avionics software performed successfully in the laboratory but experienced significant problems that delayed the program and increased costs once it entered actual flight testing in the F-22A aircraft. JSF software and advanced electronics are more complex than the F-22A's. While DOD believes it can manage program risk by holding regular program reviews, DOD's own experience has shown this approach is not effective. Accepting and managing risk instead of capturing technology, design, and manufacturing knowledge, as suggested in DOD policy, has made it difficult for DOD to make informed decisions at key points. DOD partially concurred with our recommendation to the Secretary of Defense to plan an incremental acquisition strategy that follows the intent of DOD evolutionary acquisition policy and delivers a first increment that limits new content to proven technologies and design and to report this plan to Congress by July 2006. It stated the JSF Acquisition Strategy fully complies with policy and is a knowledge- based, incremental approach that includes three blocks of increasing degrees of warfighter capability. We believe DOD's strategy does not provide the benefits of an evolutionary approach, as suggested by DOD's policy and best practices. In commenting on our report, DOD did not address the salient points concerning the true significance of adopting an incremental acquisition approach--reducing program risk, delivering needed capabilities to the warfighter quicker, and recapitalizing the aging tactical aircraft fleet sooner. The JSF strategy resembles other past major acquisition programs that have attempted to achieve the ultimate capability in a single development increment, producing nearly all of the fleet with the fully required capability from the outset. DOD has allowed technology development to spill over into product development, weakening the foundation for program cost or schedule estimates. This has led to disastrous outcomes for other major acquisition programs in the past. We continue to believe the successful F-16 program can serve as a model for the JSF program. The F-16 program evolved capabilities over a 30-year period, buying substantial quantities of each increment in order to recapitalize aging tactical aircraft and provide new capabilities to the warfighter more quickly. If DOD were to make smaller, more manageable commitments in capabilities, it would make cost and schedule more predictable and deliver needed capabilities to the warfighter sooner. We are sending copies of this report to the Secretary of Defense; the Secretaries of the Air Force, Army, and Navy; and the Director of the Office of Management and Budget. We will also provide copies to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other staff making key contributions to this report were Michael Hazard, Assistant Director; Lily Chin; Matthew Drerup; Matthew Lea; Gary Middleton; Karen Sloan; and Adam Vodraska. Signed by: Michael J. Sullivan: Director: Acquisition and Sourcing Management: List of Congressional Committees: The Honorable John Warner: Chairman: The Honorable Carl Levin: Ranking Minority Member: Committee on Armed Services: United States Senate: The Honorable Ted Stevens: Chairman: The Honorable Daniel K. Inouye: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Duncan L. Hunter: Chairman: The Honorable Ike Skelton: Ranking Minority Member: Committee on Armed Services: House of Representatives: The Honorable C.W. Bill Young: Chairman: The Honorable John P. Murtha: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Scope and Methodology: To determine if the Joint Strike Fighter (JSF) program will capture critical product knowledge before making production investments, we compared the program's plans and results to date against best practice standards for applying knowledge to support major program investment decisions. The best practice standards are based on a GAO body of work that encompasses nearly 10 years and visits to over 25 major commercial companies. Our work has shown that valuable lessons can be learned from the commercial sector, as well as successful Department of Defense (DOD) cases, and can be applied to the development of weapon systems. We reviewed JSF management reports, acquisition plans, test plans, risk assessments, cost reports, independent program assessments, and program status briefings. We identified gaps in product knowledge at the production decision points, reasons for these gaps, and the risks to program cost and schedule associated with moving forward. We reviewed DOD's acquisition policy to determine whether JSF's approach met its framework and intent. We interviewed officials from the DOD acquisition program management office and prime contractor to gain their perspectives on program risks and their approaches to managing risks. To evaluate whether the current acquisition plan follows an evolutionary or incremental approach, a key best practice for meeting business case goals, we examined the JSF program's acquisition framework and the actions taken by DOD to address the recommendations made in our 2005 report. We compared the JSF program's approach with the approaches used by leading companies and successful DOD programs to evolve products to their ultimate capabilities. We also reviewed DOD guidance on structuring evolutionary acquisition programs. To determine the JSF program outcomes to date, we used the program estimates that justified the program when it started in 1996. This was the point JSF transitioned from a technology development environment to an acquisition program environment, with the commitment to deliver a family of strike aircraft that meets the Air Force, Navy, and Marine Corps needs. At that time, total production, acquisition, and ownership costs had not been estimated. The total production, acquisition, and ownership cost estimates were first established to support the decision to enter the system development and demonstration phase in 2001. We used these estimates as the baseline for these costs. We also obtained information on past DOD programs from Selected Acquisition Reports and prior work conducted by GAO over the past 25 years. To identify opportunities to improve JSF program outcomes, we assessed the potential of applying an incremental development approach to the JSF acquisition program based on the commercial model and DOD guidance. We also examined the program history of the F-16 fighter, the JSF predecessor, which applied an incremental approach over its 30-year acquisition. The F-16 acquisition has been touted by DOD and others as a successful program and a model for others to follow. To examine the program history of the F-16, we met with F-16 program officials and analyzed acquisition plans, management reports, and program outcomes. In performing our work, we obtained information and interviewed officials from the JSF Joint Program Office, Arlington, Virginia; Lockheed Martin Aeronautical Systems, Fort Worth, Texas; Defense Contract Management Agency, Fort Worth, Texas; and offices of the Director, Operational Test and Evaluation, and Acquisition, Technology and Logistics, Program Analysis and Evaluation-Cost Analysis Improvement Group, which are part of the Office of Secretary of Defense in Washington, D.C. We also obtained information from the F-16 program office and Rand Corporation, Santa Monica, California. [End of section] Appendix II: Changes in JSF Program Purchase Quantities and Costs: Expected quantities: Development quantities; November 1996 (program start): 10; October 2001 (system development start): 14; As of December 2005: 15; Changes since initial estimate: 50. Procurement quantities (U.S. only); November 1996 (program start): 2,978; October 2001 (system development start): 2,852; As of December 2005: 2,443; Changes since initial estimate: (18). Total quantities; November 1996 (program start): 2,988; October 2001 (system development start): 2,866; As of December 2005: 2,458; Changes since initial estimate: (18). Cost estimates (then year dollars in billions): Development; November 1996 (program start): $24.8; October 2001 (system development start): $34.4; As of December 2005: $45.7; Changes since initial estimate: 84. Procurement; November 1996 (program start): Not available; October 2001 (system development start): $196.6; As of December 2005: $210.7; Changes since initial estimate: 7. Other; November 1996 (program start): Not available; October 2001 (system development start): $2.0; As of December 2005: $.2; Changes since initial estimate: (90). Total program acquisition; November 1996 (program start): Not available; October 2001 (system development start): $233.0; As of December 2005: $256.6; Changes since initial estimate: 10. Unit cost estimates (then year dollars in millions): Average procurement; November 1996 (program start): Not available; October 2001 (system development start): $69; As of December 2005: $86; Changes since initial estimate: 25. Program acquisition; November 1996 (program start): Not available; October 2001 (system development start): $81; As of December 2005: $104; Changes since initial estimate: 28. Unit recurring flyaway costs (fiscal year 2002 dollars in millions): Conventional takeoff and landing; November 1996 (program start): $31.5; October 2001 (system development start): $37.0; As of December 2005: $44.5; Changes since initial estimate: 41. Short takeoff and vertical landing; November 1996 (program start): $33.7-39.3; October 2001 (system development start): $45.8; As of December 2005: $58.7; Changes since initial estimate: 49-74. Carrier; November 1996 (program start): $34.9-42.7; October 2001 (system development start): $47.8; As of December 2005: $61.7; Changes since initial estimate: 44-77. Source: GAO analysis of DOD data: [End of table] [End of section] Appendix III: Comments from the Department of Defense: OFFICE OF THE UNDER SECRETARY OF DEFENSE: 3000 DEFENSE PENTAGON: ACQUISITION TECHNOLOGY AND LOGISTICS: WASHINGTON. DC 20301-3000: FEB 28 2006: Mr. Michael J. Sullivan: Director, Acquisition and Sourcing Management: U.S. Government Accountability Office: 441 G Street, N. W.: Washington, D.C. 20548: Dear Mr. Sullivan: This is the Department of Defense (DoD) response to the Government Accountability Office (GAO) draft report 06-356, "JOINT STRIKE FIGHTER: DOD Plans to Enter Production before Testing Demonstrates Acceptable Performance," dated January 26, 2006 (GAO Code 120450). The Department of Defense partially concurs with the three recommendations in the draft report. Details of the partial concurrences are contained in the enclosure. The Department appreciates the opportunity to comment on the draft report. Sincerely, Signed by: Mark D. Schaeffer: Acting Director: Defense Systems: Enclosure: As stated: GAO DRAFT REPORT - DATED JANUARY 26, 2006: GAO CODE 120450/GAO-06-356: "JOINT STRIKE FIGHTER: DOD Plans to Enter Production before Testing Demonstrates Acceptable Performance" DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS: RECOMMENDATION 1: The GAO recommended that the Secretary of Defense delay the investment in production aircraft until sufficient testing has at least demonstrated the basic airframe design of each JSF variant in important parts of the flight envelope, including combinations of critical altitudes, speeds, and angles of attack. (p. 22/GAO Draft Report): DOD RESPONSE: Partially Concur. The Department agrees that a knowledge- based approach is critical to making prudent acquisition decisions. The JSF Acquisition Strategy incorporates this type of approach. Established decision criteria to support each production decision include specific test deliverables. The JSF is in the 5th year of a 12-year development effort. In 2004, the program was restructured to allow additional development time to meet performance requirements associated with the Short Takeoff and Vertical Landing (STOVL) variant. Due to the high degree of commonality throughout the three JSF variants, the design changes from the STOVL weight reduction effort benefit the Conventional Takeoff and Landing (CTOL) and Carrier Variant (CV). The restructured program extended the System Development and Demonstration (SDD) phase by 18 months. The Department approved an updated Acquisition Strategy in May 2005 that reflects changes since the start of SDD, including the restructuring. A significant aspect of the restructuring was a comprehensive review and update to integrated test and evaluation planning. The Department is confident that the Acquisition Strategy and management practices currently in place achieve the objectives of GAO's recommendation. RECOMMENDATION 2: The GAO recommended that the Secretary of Defense limit production quantities to the number of aircraft needed for operational testing and that can be produced using the current production capability (based on existing tooling, materials, and personnel) until a fully integrated aircraft (with advanced mission systems and predictive maintenance systems) demonstrates through flight testing the required capabilities. (p. 22/GAO Draft Report): DOD RESPONSE: Partially Concur. The Department plans limited production until aircraft capabilities are fully demonstrated. Further limitations on production are not necessary. To reduce both risk and cost, the JSF Acquisition Strategy takes advantage of the benefits of flight test and significant advances in modeling, simulation, and lab testing that have been achieved in the commercial and military sectors. The JSF Acquisition Strategy provides the most effective balance of technical risk, financial resources, and the Services' operational needs. The JSF Acquisition Strategy includes clear entry and exit criteria for critical milestones to ensure that technologies are mature, and required incremental test objectives are achieved before obligating funds for SDD or production. The Department conducts acquisition reviews via Integrating Integrated Product Teams and Overarching Integrated Product Teams, which support Defense Acquisition Board reviews. Configuration Steering Board and Service Acquisition Executive reviews are conducted quarterly to assess program performance, including test objectives, ensuring associated program risks are understood and appropriately mitigated. RECOMMENDATION 3: The GAO recommended that the Secretary of Defense plan an incremental acquisition strategy that follows the intent of DOD acquisition policy and delivers a first increment that limits new content to proven technologies and design. The plan should be completed and reported to the Congress by July 2006, and include adjustments in associated programs as necessary to maintain needed capabilities. (p. 22/GAO Draft Report): DOD RESPONSE: Partially Concur. The Department's approved JSF Acquisition Strategy fully complies with acquisition policy. It is premised on a knowledge-based, incremental approach that provides increasing degrees of warfighting capability with each block. Technical maturity, test accomplishments and operational effectiveness are key considerations at each acquisition decision point. Early production aircraft will have Block 1 capability for standup of initial training squadrons. Block 2 upgrades will support Marine Corps Initial Operational Capability. Block 3 will provide full avionics and key weapons capability. Previously delivered Block 1 and 2 aircraft will be upgraded to Block 3 capability. This approach provides initial, incremental capability to meet the most immediate, achievable warfighting requirements while development on more complex capabilities continues. The Department structured the JSF program from the beginning to emphasize jointness, technology maturation and concept demonstrations, as well as early cost and performance trades integral to the weapons system requirements definition process. The Department approved the updated JSF Acquisition Strategy in May 2005. Department and JSF Program leadership have provided program updates to Congress since then and will continue to do so. [End of section] FOOTNOTES [1] Section 213 of the act requires us to assess the extent to which the system development and demonstration program is currently meeting key cost, schedule, and performance goals; the likelihood that the program will be completed within estimated costs; and the program's current acquisition plan leading to production. [2] GAO, Tactical Aircraft: Opportunity to Reduce Risks in Joint Strike Fighter Program with Different Acquisition Strategy, GAO-05-271 (Washington, D.C.: Mar. 15, 2005). [3] A business case provides demonstrated evidence that (1) the warfighter need exists and that it can best be met with the chosen concept and (2) the concept can be developed and produced within existing resources--including proven technologies, design knowledge, adequate funding, and adequate time to deliver the product when needed. Establishing a business case calls for a realistic assessment of risks and costs; doing otherwise undermines the intent of the business case and invites failure. [4] A production representative aircraft is one that is built using production-like tooling and methods and using production workers and is of the design expected be built to satisfy the requirements of the warfighter. Flight-testing a production representative prototype that incorporates all critical software is the primary method to capture key design knowledge that indicates the system will work as planned, is reliable, and is ready for production. The higher the percentage of testing completed, the lower the risk of investing in production. [5] Sensor fusion is the ability to take information from both multiple onboard and offboard aircraft sensors and display the information in an easy-to-use format for the pilot. It is vitally important, because the JSF is a single-seat aircraft, and the pilot needs support to carry out multiple types of missions. [6] Initial operational testing consists of field tests intended to demonstrate a system's effectiveness and suitability for military use. [7] These figures do not include the potential for orders for international partners during low-rate initial production. Preliminary data indicate that these orders could significantly increase this rate. [8] DOD policy contains three critical milestone decision points in its acquisition process. One of these, referred to as milestone C, authorizes entry into low-rate initial production. However, the JSF program is deviating from that milestone framework and has delayed this critical milestone decision point to the full-rate production decision point in 2013--more than 6 years after entering low-rate initial production. This delay compromises the controls expected in the DOD acquisition policy for the start of production. [9] During this approximately 6-month period of initial tests, which includes about 30 planned flights, the program intends to demonstrate the aircraft's flight qualities in limited flight conditions, takeoff and landing, and initial air refueling. However, recent program data indicate that the first flight may be delayed until October 2006, reducing the number of flights that may be completed before the production decision. [10] Rand Corporation, Test and Evaluation Trends and Costs for Aircraft and Guided Weapons (Santa Monica, California, 2004). [11] These technologies are not expected to be fully mature until 2011. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: