This is the accessible text file for GAO report number GAO-10-227SP entitled 'NASA: Assessments of Selected Large-Scale Projects' which was released on February 1, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: February 2010: NASA: Assessments of Selected Large-Scale Projects: GAO-10-227SP: GAO Highlights: Highlights of GAO-10-227SP, a report to congressional committees. Why GAO Did This Study: The National Aeronautics and Space Administration (NASA) plans to invest billions in the coming years in science and exploration space flight initiatives. The scientific and technical complexities inherent in NASA's mission create great challenges in managing its projects and controlling costs. In the past, NASA has had difficulty meeting cost, schedule, and performance objectives for many of its projects. The need to effectively manage projects will gain even more importance as NASA seeks to manage its wide-ranging portfolio in an increasingly constrained fiscal environment. This report provides an independent assessment of selected NASA projects. In conducting this work, GAO compared projects against best practice criteria for system development including attainment of knowledge on technologies and design. GAO also identified other programmatic challenges that were contributing factors in cost and schedule growth of the projects reviewed. The projects assessed are considered major acquisitions by NASA—each with a life-cycle cost of over $250 million. No recommendations are provided in this report; however, GAO has reported extensively and made recommendations on NASA acquisition management in the past. GAO has designated NASA's acquisition management as a high risk area since 1990. What GAO Found: GAO assessed 19 NASA projects with a combined life-cycle cost of more than $66 billion. Of those 19 projects, 4 are still in the formulation phase where cost and schedule baselines have yet to be established, and 5 just entered the implementation phase in fiscal year 2009 and therefore do not have any cost and schedule growth. However, 9 of the 10 projects that have been in the implementation phase for several years experienced cost growth ranging from 8 to 68 percent, and launch delays of 8 to 33 months, in the past 3 years. These 10 projects had average development cost growth of almost $121.1 million—or 18.7 percent—and schedule growth of 15 months, and a total increase in development cost of over $1.2 billion, with over half of this total—or $706.6 million—occurring in the last year. In some cases, cost growth was higher than is reported because it occurred before project baselines were established in response to the statutory requirement in 2005 for NASA to report cost and schedule baselines for projects and implementation with estimated life-cycle cost of more than $250 million. See the table below for a summary of the 10 projects. Additionally, NASA was recently appropriated over $1 billion through the American Recovery and Reinvestment Act of 2009. Table: Cost and Schedule Growth of Selected NASA Projects in the Implementation Phase: Project: Aquarius[A]; Baseline (FY): 2008; Development cost growth: $15.9 million; Percent cost growth: 8.3%; Launch delay (Months): 10. Project: Glory[A]; Baseline (FY): 2009; Development cost growth: $37.0 million; Percent cost growth: 14.3%; Launch delay (Months): 16. Project: Herschel[A]; Baseline (FY): 2007; Development cost growth: $9.7 million; Percent cost growth: 8.3%; Launch delay (Months): 21. Project: Kepler[A]; Baseline (FY): 2007; Development cost growth: $77.5 million; Percent cost growth: 24.8%; Launch delay (Months): 9. Project: LRO[A]; Baseline (FY): 2008; Development cost growth: $52.3 million; Percent cost growth: 12.4%; Launch delay (Months): 8. Project: MSL[A]; Baseline (FY): 2008; Development cost growth: $662.4 million; Percent cost growth: 68.4%; Launch delay (Months): 25. Project: NPP[A]; Baseline (FY): 2007; Development cost growth: $132.1 million; Percent cost growth: 22.3%; Launch delay (Months): 33. Project: SDO[A]; Baseline (FY): 2007; Development cost growth: $58.9 million; Percent cost growth: 9.4%; Launch delay (Months): 18. Project: SOFIA[A]; Baseline (FY): 2007; Development cost growth: $162.3 million; Percent cost growth: 17.7%; Launch delay (Months): 12. Project: WISE; Baseline (FY): 2008; Development cost growth: $2.8 million; Percent cost growth: 1.5%; Launch delay (Months): 1. Project: Average; Development cost growth: $121.1 million; Percent cost growth: 18.7%; Launch delay (Months): 15. Project: Total cost growth; Development cost growth: $1,210.9 million. Source: GAO analysis of NASA project data. [A] Indicates projects that exceeded their cost and/or schedule baselines. [End of table] Many of the projects GAO reviewed experienced challenges in developing new or retrofitting older technologies, stabilizing engineering designs, managing the performance of their contractors and development partners, as well as funding and launch planning issues. Reducing the kinds of problems this assessment identifies in acquisition programs hinges on developing a sound business case for a project. Based, in part, on GAO's previous recommendations, NASA has acted to adopt practices that would ensure programs proceed based on a sound business case and undertaken initiatives aimed at improving program management, cost estimating, and contractor oversight. Continued attention to these efforts and effective, disciplined implementation should help maximize NASA's acquisition investments. To view the full product, including the scope and methodology, click on [hyperlink, http://www.gao.gov/products/GAO-10-227SR]. For more information, contact Cristina Chaplain at (202) 512-4841 or chaplainc@gao.gov. [End of section] Contents: Forward: Letter: A Sound Business Case Underpins Successful Acquisition Outcomes: NASA Continues to Improve Its Acquisitions: Our Observations: Project Challenges: Project Assessments: Aquarius: Ares I Crew Launch Vehicle (CLV): Glory: Global Precipitation Measurement (GPM) Mission: Gravity Recovery and Interior Laboratory (GRAIL): Herschel: Juno: James Webb Space Telescope (JWST): Kepler: Landsat Data Continuity Mission (LDCM): Lunar Reconnaissance Orbiter (LRO): Magnetospheric Multiscale (MMS): Mars Science Laboratory (MSL): NPOESS Preparatory Project (NPP): Orion Crew Exploration Vehicle (CEV): Radiation Belt Storm Probes (RBSP): Solar Dynamics Observatory (SDO): Stratospheric Observatory for Infrared Astronomy (SOFIA): Wide-field Infrared Survey Explorer (WISE): Agency Comments and Our Evaluation: Appendixes: Appendix I: Comments from the National Aeronautics and Space Administration: Appendix II: Objectives, Scope, and Methodology: Appendix III: NASA Life Cycle for Flight Systems Compared to a Knowledge-Based Approach: Appendix IV: Technology Readiness Levels: Appendix V: NASA Projects Receiving Additional Funding: Appendix VI: GAO Contact and Staff Acknowledgments: Tables: Table 1: Cost and Schedule Growth of Selected NASA Projects in the Implementation Phase: Table 2: Assessment of Challenges for Selected NASA Projects: Table 3: Schedule Growth for Projects with and without Partners: Table 4: ARRA Funding for Review NASA Projects: Figures: Figure 1: Summary of Projects Assessed by Phase of the NASA Project Life Cycle: Figure 2: Illustration of Project Two-Page Summary: Figure 3: NASA's Life Cycle for Flight Systems Compared to a Knowledge- Based Approach: Abbreviations: AFB: Air Force Base: AFS: Air Force Station: AIA: Atmospheric Imaging Assembly: APS: aerosol polarimetry sensor: ARRA: American Recovery and Reinvestment Act of 2009: ASI: Argenzia Spaciale Italiana (Italian Space Agency): C&DH: Command and Data Handling: CCD: charged-coupled device: CDDS: Cavity Door Drive System: CDR: critical design review: CEV: Crew Exploration Vehicle: CLV: Crew Launch Vehicle: CNES: Centre National d'Etudes Spatiales: CONAE: Comision Nacional de Actividades Espaciales (Space Agency of Argentina): CrIS: Cross-track Infrared Sounder: CSA: Canadian Space Agency: CSL: Centre Spatial de Liege: DCI: data collection instrument: DLR: German Aerospace Center: DPR: dual-frequency precipitation radar: ESA: European Space Agency: EVE: Extreme Ultraviolet Variability Experiment: FSSA: Fine Sun Sensor Assembly: GMI: GPM microwave imager: GPM: Global Precipitation Measurement (mission): GRACE: Gravity Recovery and Climate Experiment: GRAIL: Gravity Recovery and Interior Laboratory: HIFI: Heterodyne Instrument for the Far Infrared: HIPO: High-speed Imaging Photometer for Occultation: HMI: Helioseismic and Magnetic Imager: HOPE: Helium-Oxygen-Proton-Electron: IPO: Integrated Program Office: JAXA: Japanese Aerospace Exploration Agency: JCL: Joint Cost and Schedule Confidence Levels: JPL: Jet Propulsion Laboratory: JWST: James Webb Space Telescope: KDP: key decision point: LCROSS: Lunar Crater Observation and Sensing Satellite: LDCM: Landsat Data Continuity Mission: LIO: Low Inclination Observatory: LRO: Lunar Reconnaisance Orbiter: MEP: Mars Exploration Program: MMS: Magnetospheric Multiscale: MSL: Mars Science Laboratory: NAR: nonadvocate review: NASA: National Aeronautics and Space Administration: NID: NASA Interim Directive: NPR: NASA Procedural Requirements: NPOESS: National Polar-Orbiting Operational Environmental Satellite System: NPP: NPOESS Preparatory Project: OCO: Orbiting Carbon Observatory: OLI: Operational Land Imager: PA&E: Office of Program Analysis and Evaluation (NASA): PDR: preliminary design review: PICA: phenolic impregnated carbon ablator: RBSP: Radiation Belt Storm Probes: SBC: single board computer: SDO: Solar Dynamics Observatory: SDR: system definition review: SOFIA: Stratospheric Observatory for Infrared Astronomy: SPIRE: Spectral and Photometric Imaging Receiver: TIM: total irradiance monitor: TLGA: Toroidal Low Gain Antenna: TMDS: Thermal Mass Dynamics Simulator: TRL: technology readiness level: ULA: United Launch Alliance: USGS: U.S. Geological Service: VIERS: Visible Infrared Imaging Radiometer Suite: WISE: Wide-field Infrared Survey Explorer: [End of section] United States Government Accountability Office: Washington, DC 20548: February 1, 2010: Congressional Committees: I am pleased to present GAO's second annual assessment of selected large-scale NASA projects. This report provides a snapshot of how well NASA plans and executes major acquisitions—a topic that has been on GAO's High-Risk List since the list's inception in 1990. In this year's report, we found that NASA frequently exceeded its own acquisition cost and schedule estimates, even when those estimates were relatively new In fact, 9 out of 10 projects that have been in implementation for several years significantly exceeded their cost or schedule baseline estimates—all in the last 3 years. NASA's ongoing struggle to meet budget and schedule demands comes at a time when the agency is on the verge of major changes. The Space Shuttle is slated to retire this year after nearly 30 years of service, the International Space Station draws closer to its scheduled retirement in 2016, and a new means of human space flight is under development, and the very future of which has been hotly debated and recently reviewed by an independent commission, and awaits a presidential decision. Amid all this change, one thing that will remain constant is NASA's need to manage programs and projects with a budget that has remained relatively constant in recent years. This will require hard choices among competing priorities within the organization, which must balance its core missions in science, aeronautics, and human space flight and exploration. In addition, NASA will be competing for an ever-shrinking share of discretionary spending against other national priorities, such as the economy, combating terrorism, and health care reform. We believe that this report can provide insights that will help NASA place programs in a better position to succeed and help the agency maximize its investments. Our work has shown that reducing the project challenges that can lead to cost and schedule growth this report identifies hinges on developing a sound business case that includes firm requirements, mature technologies, a knowledge-based acquisition strategy, realistic cost estimates, and sufficient funding. To its credit, NASA has continued to take steps to improve its acquisition process along these lines. The revisions aim to provide key decision- makers with increased knowledge needed to make informed decisions before a program starts, and to maintain discipline once it begins. Implementation of these revisions, however, will require senior NASA leaders to have the will to terminate projects that do not measure up, to recognize and reward savings, and to hold appropriate parties accountable for poor outcomes. Signed by: Gene L. Dodaro: Acting Comptroller General of the United States: [End of section] United States Government Accountability Office: Washington, DC 20548: February 1, 2010: Congressional Committees: The National Aeronautics and Space Administration's (NASA) portfolio of major projects ranges from highly complex and sophisticated space transportation vehicles, to robotic probes, to satellites equipped with advanced sensors to study the earth. In many cases, NASA's projects are expected to incorporate new and sophisticated technologies that must operate in harsh, distant environments. These projects have also produced ground-breaking research and advanced our understanding of the universe. However, one common theme binds most of the projects—they cost more and take longer to develop than planned. We reported last year that 10 out of 13 NASA projects experienced significant cost and/or schedule growth from baselines established only 2 or 3 years earlier.[Footnote 1] For example, the Glory project, a science satellite designed to help understand how the sun and particles in the atmosphere affect Earth's climate, saw its development costs increase more than 50 percent—from $169 to $259 million—since 2008. Congress reauthorized[Footnote 2] the Glory project in fiscal year 2009 and new cost and schedule baselines were then established. Similarly, technical issues delayed the Mars Science Laboratory by 2 years, and the project, which was already over budget, is now scheduled to cost over $660 million more than estimated in 2007— an increase of over 68 percent in development costs. In prior years, programs such as the X-33 and X-34, which were meant to demonstrate technology for future reusable launch vehicles, were canceled because of technical difficulties and cost overruns after NASA spent more than $1 billion on the programs. NASA acknowledges the problem and is striving to improve its cost estimating and program execution. The agency notes that most missions are one of a kind and complex and that external factors, such as launch scheduling and spotty performance by development partners, also cause delays and cost increases. Although space development programs are complex and difficult by nature, our work consistently finds that inherent risks are exacerbated by poor acquisition management. Moreover, the reality of cost and schedule increases can have secondary impacts when projects that are seemingly on track end up being the bill-payer for troubled projects. This also makes it hard to manage the portfolio and make investment decisions. Congress has expressed concern about NASA's performance and has identified the need to standardize the reporting of cost, schedule, and content for NASA research and development projects. In 2005, Congress required NASA to report cost and schedule baselines— benchmarks against which changes can be measured—for all NASA programs and projects with estimated life-cycle costs of at least $250 million that have been approved to proceed to the development stage, known as implementation,[Footnote 3] in which components begin to take physical form. It also required that NASA report to Congress when development cost is likely to exceed the baseline estimate by 15 percent or more, or when a milestone is likely to be delayed beyond the baseline estimate by 6 months or more.[Footnote 4] In response, NASA began establishing cost and schedule baselines in 2006 and has been using them as the basis for annual project performance reports to the Congress provided in its annual budget submission each year. While establishing the baselines required by the Congress enabled a more consistent reporting among NASA projects, it also made past cost and schedule growth less transparent. Consequently, the cost and schedule breaches presented in this report represent only increases from the baselines established after the 2005 congressional requirement. Recently, NASA was appropriated over $1 billion through the American Recovery and Reinvestment Act of 2009[Footnote 5] to help spur technological advances in science. The agency's Science and Exploration Systems Mission Directorates were each appropriated $400 million under this supplemental appropriation. As of October 2009, the projects covered in this assessment are scheduled to receive $470 million as a part of the total allocation that NASA intends to use to assist with such items as developing instruments and spacecraft, maintaining the current workforce, and building test facilities. Appendix V provides a listing of NASA projects in our review receiving funding under the American Recovery and Reinvestment Act of 2009 and the intended use of those funds for each project. The explanatory statement of the House Committee on Appropriations accompanying the Fiscal Year 2009 Omnibus Appropriations Act directed GAO to prepare project status reports on selected large-scale NASA programs, projects, or activities. This report responds to that mandate by assessing 19 NASA projects, each with an estimated life- cycle cost over $250 million. The combined estimated life-cycle cost for these 19 projects exceeds $66 billion. Each assessment is presented in a two-page summary that analyzes the project's cost and schedule status and project challenges we identified with the objective to identify risks that, if mitigated, could put NASA in a better position to succeed. We also provide general observations about the performance of NASA's major projects and the agency's management of those projects during development. In doing so, the report expands on the importance of developing a knowledge-based acquisition strategy and to provide decision-makers with an independent, knowledge-based assessment of individual systems that identifies potential risks and allows the decision-makers to take actions to put projects that are early in the development cycle in a better position to succeed. NASA provided updated cost and schedule data as of October 2009 for 14 of the 19 projects.[Footnote 6] We reviewed and compared that data to previously established cost and schedule baselines for each of those 14 projects. We took appropriate steps to address data reliability. Our approach included an examination of the current phase of a project's development and how each project was advancing. Each project we reviewed was in either the formulation phase or the implementation phase of the project life cycle. In the formulation phase, the project defines requirements—what the project is being designed to do—matures technology, establishes a schedule, estimates costs, and produces a plan for implementation. In the implementation phase, the project carries out these plans, performing final design and fabrication as well as testing components and system assembly, integrating these components and testing how they work together, and launching the project. This phase also includes the period from project launch through mission completion. We assessed each project's cost and schedule and characterized growth in either as significant if it exceeded the baselines that trigger reporting to the Congress under the law.[Footnote 7] Based on our previous reviews and discussions with project officials and drawing on GAO's established criteria for knowledge-based acquisitions and on other GAO work on system acquisitions, we identified six challenges that can contribute to cost and schedule growth in these projects: technology maturity, design stability, contractor performance, development partner performance, funding issues, and launch manifest issues. This list of challenges is not exhaustive, and we believe these challenges will evolve as we continue this work into the future. To assess technology maturity, we examined the projects' reported critical technology readiness levels—a measure that NASA devised and that is now used at other agencies as well. We looked at the technology readiness level at the time of the project's preliminary design review, which occurs just before it enters the implementation phase, and compared that against the level of maturity that best practices call for at that stage to minimize risks. Based in part on our discussions with officials for the individual projects and data submitted by the projects, we identified the extent to which project cost and schedule were negatively impacted by challenges integrating heritage—or preexisting— technology into their projects. To assess design stability, we examined the percentage of engineering drawings completed or projected to be completed by the critical design review—which is usually held about midway through the project's development. We asked project officials to provide this information, and we compared it against GAO's best practices' metric of 90 percent of drawings released by the critical design review. We also discussed the extent to which contractors' and development partners' challenges in developing and delivering project hardware affected overall project cost and schedule. To assess funding issues, we interviewed project officials and reviewed budget documents to determine if increases to cost or schedule resulted from interrupted or delayed funding, or if project officials indicated that the project had poor phasing of the project's funding plan. To assess launch manifest issues, we interviewed launch services officials to determine what projects had to reschedule launch dates based on an inability to be ready for launch or other factors. The individual project offices were given an opportunity to provide comments and technical clarifications on our assessments prior to their inclusion in the final product. We conducted this performance audit from April 2009 to February 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II contains detailed information on our scope and methodology. We do not provide recommendations in this report. A Sound Business Case Underpins Successful Acquisition Outcomes Many of NASA's projects are one-time articles, meaning that there is little opportunity to apply knowledge gained to the production of a second, third, or future increments of spacecraft. In addition, NASA often partners with other domestic partners and other space-faring countries, including several European nations, Japan, and Argentina. These partnerships go a long way to foster international cooperation in space, but they also subject NASA projects to added risk such as when partners do not meet their obligations or run into technical obstacles they cannot easily overcome. While space development programs are complex and difficult by nature, and most are one-time efforts, the nature of its work should not preclude NASA from achieving what it promises when requesting and receiving funds. We have reported that NASA would benefit from a more disciplined approach to its acquisitions. The development and execution of a knowledge-based business case for these projects can provide early recognition of challenges, allow managers to take corrective action, and place needed and justifiable projects in a better position to succeed. Our studies of best practice organizations show the risks inherent in NASA's work can be mitigated by developing a solid, executable business case before committing resources to a new product development.[Footnote 8] In its simplest form, this is evidence that (1) the customer's needs are valid and can best be met with the chosen concept, and (2) the chosen concept can be developed and produced within existing resources—that is, proven technologies, design knowledge, adequate funding, and adequate time to deliver the product when needed. A program should not go forward into product development unless a sound business case can be made. If the business case measures up, the organization commits to the development of the product, including making the financial investment. Our best practice work has shown that developing business cases based on matching requirements to resources before program start leads to more predictable program outcomes—that is, programs are more likely to be successfully completed within cost and schedule estimates and deliver anticipated system performance.[Footnote 9] At the heart of a business case is a knowledge-based approach to product development that is a best practice among leading commercial firms. Those firms have created an environment and adopted practices that put their program managers in a good position to succeed in meeting expectations. A knowledge-based approach requires that managers demonstrate high levels of knowledge as the program proceeds from technology development to system development and, finally, production. In essence, knowledge supplants risk over time. This building of knowledge can be described over the course of a program, as follows: * When a project begins development, the customer's needs should match the developer's available resources—mature technologies, time, and funding. An indication of this match is the demonstrated maturity of the technologies needed to meet customer needs—referred to as critical technologies. If the project is relying on heritage—or pre-existing— technology, that technology must be in appropriate form, fit, and function to address the customer's needs within available resources. The project will normally enter development after completing the preliminary design review, at which time a business case should be in hand. * Then, about midway through the product's development, its design should be stable and demonstrate it is capable of meeting performance requirements. The critical design review takes place at that point in time because it generally signifies when the program is ready to start building production-representative prototypes. If design stability is not achieved, but a product development continues, costly re-designs to address changes to project requirements and unforeseen challenges can occur. By the critical design review, design should be stable and capable of meeting performance requirements. * Finally, by the time of the production decision, the product must be shown to be producible within cost, schedule, and quality targets and have demonstrated its reliability, and the design must demonstrate that it performs as needed through realistic system-level testing. Lack of testing increases the possibility that project managers will not have information that could help avoid costly system failures in late stages of development or during system operations. Our best practices work has identified numerous other actions that can be taken to increase the likelihood that a program can be successfully executed once that business case is established. These include ensuring cost estimates are complete, accurate, and updated regularly and holding suppliers accountable through such activities as regular supplier audits and performance evaluations of quality and delivery. Moreover, we have recommended using metrics and controls throughout the life cycle to gauge when the requisite level of knowledge has been attained and when to direct decision makers to consider criteria before advancing a program to the next level and making additional investments. The consequence of proceeding with system development without establishing and adhering to a sound business case is substantial. GAO and others have reported that NASA has experienced cost and schedule growth in several of its projects over the past decade, resulting from problems that include failing to adequately identify requirements and underestimating complexity and technology maturity. We have found that the need to meet schedule is one of the main reasons why programs cannot execute as planned. Short cuts, such as developing technology while design work and construction are already underway, and delaying or reducing tests, are taken to meet schedule. Ultimately, when a schedule is set that cannot accommodate the work that needs to be done, costs go up and capability is delayed. Delaying the delivery of these capabilities can also have a ripple effect throughout NASA projects as staff must then stay on a given project longer than intended, thus increasing the project's costs, and crippling other projects that had counted on using newly available staff to move forward. NASA Continues Efforts to Improve Its Acquisitions: In 2005, we reported that NASA's acquisition policies did not conform to best practices for product development because those policies lacked major decision reviews at several key points in the project life-cycle that would allow decision makers to make informed decisions about whether a project should be authorized to proceed in the development life cycle. Based in part on our recommendations, NASA issued a revised policy in March 2007[Footnote 10] that institutes several key decision points (KDP) in the development life cycle for space flight programs and projects. At each KDP, a decision authority is responsible for authorizing the transition to the next life-cycle phase for the project.[Footnote 11] In addition, NASA's acquisition policies also require that technologies be sufficiently mature at the preliminary design review before the project enters implementation, that the design is appropriate to support proceeding with full-scale fabrication, assembly, integrating and test at the critical design review, and that the system can be fabricated within cost, schedule, and performance specifications. These changes brought the policy more in line with best practices for product development. A more detailed discussion of NASA's acquisition policy and how it relates to best practices is provided in appendix III of this report. Further, in response to GAO's designation of NASA acquisition management as a high risk area,[Footnote 12] NASA developed a corrective action plan to improve the effectiveness of NASA's program/project management.[Footnote 13] The approach focuses on how best to ensure the mitigation of potential issues in acquisition decisions and better monitor contractor performance. The plan identifies five areas for improvement—program/project management, cost reporting processes, cost estimating and analysis, standard business processes, and management of financial management systems—each of which contains targets and goals to measure improvement. As part of this initiative, NASA has taken a positive step to improve management oversight of project cost, schedule, and technical performance with the establishment of a baseline performance review reporting to NASA's senior management. Through monthly reviews, NASA intends to highlight projects that are predicted to exceed internal NASA cost and/or schedule baselines, which are set lower than cost and schedule baselines submitted to Congress, so the agency can take preemptive actions to minimize the projects' potential cost overruns or schedule delays. During our data collection efforts, we reviewed several projects' monthly and quarterly status reports, which gave us insight into their status, risks, and issues. While this reporting structure might enable management to be aware of the issues projects are facing, it is too early to tell if the monthly reviews are having the intended impact of enabling NASA management to take preemptive cost saving actions, such as delaying a design review or canceling a project. As a part of the continuing effort to improve its acquisition processes, NASA has begun a new initiative—Joint Cost and Schedule Confidence Levels (JCL)—to help programs and projects with management, cost and schedule estimating, and maintenance of adequate levels of reserves. Under this new policy, cost, schedule, and risk are combined into a complete picture to help inform management of the likelihood of a project's success. Utilizing JCL, each project will receive a cost estimate with a corresponding confidence level—the percentage probability representing the likelihood of success at the specified funding level. NASA believes the application of this policy will help reduce the cost and schedule growth in its portfolio and improve transparency, and increase the probabilities of meeting those expectations. NASA's goal is for all projects that have entered the implementation phase to have a JCL established by spring 2010. While these efforts are positive steps, it is too early to assess their impact and they will be limited if project officials are not held accountable for demonstrating that elements of a knowledge-based business case are demonstrated at key junctures in development. For projects to have better outcomes not only must they demonstrate a high level of knowledge at key junctures, but decision makers must also use this information to determine whether and how best a project should proceed through the development life cycle. If done successfully, these measures should enable NASA to foster the expansion of a business-oriented culture, reduce persistent cost growth and schedule delays, and maximize investment dollars. Our Observations: We assessed 19 large-scale NASA projects in this review. Four of these projects were in the formulation phase where cost and schedule baselines have yet to be established, while 15 had entered implementation. Nine of the 15 projects experienced significant cost and/or schedule growth from their project baselines,[Footnote 14] while five of the remaining projects had just entered implementation and their cost and schedule baselines were established in fiscal year 2009. NASA provided cost and schedule data for 14 of the 15 projects in the implementation phase of the project life cycle. Despite being in implementation, NASA did not provide cost or schedule data for the Magnetospheric Multiscale (MMS) project. NASA will not formally release its baseline cost and schedule estimates for this project until the fiscal year 2011 budget submission to Congress, and late in our review process agency officials notified us that they will not provide project estimates to GAO until that time. NASA also did not provide formal cost and schedule information for the projects in formulation, citing that those estimates were still preliminary. See figure 1 for a summary of these projects. Figure 1: Summary of Projects Assessed by Phase of the NASA Project Life Cycle: [Refer to PDF for image: illustration] Total projects reviewed: 19; Projects in formulation: 4; Projects in implementation: 15; Projects with significant cost and/or schedule growth: 9; Projects establishing initial baselines in FY 2009, thus no cost/schedule growth: 5; Projects in implementation without significant cost/schedule growth: 1. Source: GAO analysis of NASA project data. [End of figure] Based on our analysis, development costs for projects in our review increased by an average of over 13 percent from their baseline cost estimates—including one project that increased by over 68 percent—and an average delay of almost 11 months to their launch dates. These averages were significantly higher when the four projects that just entered implementation are excluded. Specifically, there are 10 projects of analytical interest because (1) they are in the implementation phase, and (2) their baselines are old enough to begin to track variances. Most of these 10 projects have experienced significant cost and/or schedule growth, often both. These projects had an average development cost growth of 18.7 percent—or almost $121.1 million—and schedule growth of over 15 months, and a total increase in development cost of over $1.2 billion. Over half of this total increase in development cost—or $706.6 million—occurred in the last year. These cost growth and schedule delays have all occurred within the last 3 years, and a number of these projects had experienced considerable cost growth before baselines were established in response to the 2005 statutory reporting requirement.[Footnote 15] See table 1 below for the cost and schedule growth of the NASA projects in the implementation phase. Table 1: Cost and Schedule Growth of Selected NASA Projects in the Implementation Phase: Project: Aquarius[A]; Baseline (FY): 2008; Development cost growth: $15.9 million; Percent cost growth: 8.3%; Launch delay (Months): 10. Project: Glory[A]; Baseline (FY): 2009; Development cost growth: $37.0 million; Percent cost growth: 14.3%; Launch delay (Months): 16. Project: Grail; Baseline (FY): 2009; Development cost growth: $0.0; Percent cost growth: 0; Launch delay (Months): 0. Project: Herschel[A]; Baseline (FY): 2007; Development cost growth: $9.7 million; Percent cost growth: 8.3%; Launch delay (Months): 21. Project: Juno; Baseline (FY): 2009; Development cost growth: $0.0; Percent cost growth: 0; Launch delay (Months): 0. Project: JWST; Baseline (FY): 2009; Development cost growth: $0.0; Percent cost growth: 0; Launch delay (Months): 0. Project: Kepler[A]; Baseline (FY): 2007; Development cost growth: $77.5 million; Percent cost growth: 24.8%; Launch delay (Months): 9. Project: LRO[A]; Baseline (FY): 2008; Development cost growth: $52.3 million; Percent cost growth: 12.4%; Launch delay (Months): 8. Project: MSL[A]; Baseline (FY): 2008; Development cost growth: $662.4 million; Percent cost growth: 68.4%; Launch delay (Months): 25. Project: NPP[A]; Baseline (FY): 2007; Development cost growth: $132.1 million; Percent cost growth: 22.3%; Launch delay (Months): 33. Project: RBSP; Baseline (FY): 2009; Development cost growth: $0.0 million; Percent cost growth: 0; Launch delay (Months): 0. Project: SDO[A]; Baseline (FY): 2007; Development cost growth: $58.9 million; Percent cost growth: 9.4%; Launch delay (Months): 18. Project: SOFIA[A]; Baseline (FY): 2007; Development cost growth: $162.3 million; Percent cost growth: 17.7%; Launch delay (Months): 12. Project: WISE; Baseline (FY): 2008; Development cost growth: $2.8 million; Percent cost growth: 1.5%; Launch delay (Months): 1. Project: Average; Percent cost growth: 13.4%; Launch delay (Months): 11. Project: Total cost growth; Development cost growth: $1,210.9 million. Source: GAO analysis of NASA project data. [A] Indicates projects that exceeded their cost and/or schedule baselines. [End of table] Despite having baselines established in fiscal year 2008, two projects have sought reauthorization from Congress because of development cost growth in excess of 30 percent.[Footnote 16] Congress reauthorized the Glory project in fiscal year 2009,[Footnote 17] and new cost and schedule baselines were established after the project experienced a 53 percent cost growth and 6-month launch delay from original baseline estimates. The Glory project has since breached its revised schedule baseline by 16 months and exceeded its development cost baseline by over 14 percent-for a total development cost growth of over 75 percent in just 2 years. Project officials also indicated that recent technical problems could cause additional cost growth. Similarly, the Mars Science Laboratory project is currently seeking reauthorization from Congress after experiencing development cost in excess of 30 percent. Project Challenges: All six factors we assessed can lead to project cost and schedule growth: technology maturity, design stability, contractor performance, development partner performance, funding issues, and launch manifest issues. These factors—characterized as project challenges—were evident in the projects that had reached the implementation phase of the project life cycle, but many of them began in the formulation phase. We did not specifically correlate individual project challenges with specific cost and/or schedule changes in each project. The degree to which each specific challenge contributed to cost and schedule growth varied across the projects in this review and we did not assign any specific challenge as a primary factor for cost and/or schedule growth. Table 2 depicts the extent to which each of the six challenges occurred for each of the 19 projects we reviewed. Table 2: Assessment of Challenges for Selected NASA Projects: In Implementation: Project: Aquarius; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Check]; Contractor performance: [Empty]; Development partner performance: [Check]; Funding issues: [Check]; Launch manifest: [Empty]. Project: Glory; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Check]; Contractor performance: [Check]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: [Check]; Project: GRAIL; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: [Check]. Project: Herschel; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Check]; Contractor performance: [Empty]; Development partner performance: [Check]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: Juno; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Check]; Contractor performance: [Empty]; Development partner performance: [Check]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: JWST; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Project: Kepler; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Check]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Project: LRO; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: Project: MMS; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: MSL; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Check]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: NPP; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: Contractor performance: [Empty]; Development partner performance: [Check]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: RBSP; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: SDO; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Check]; Contractor performance: [Check]; Development partner performance: [Check]; Funding issues: [Check]; Launch manifest: [Check]. Project: SOFIA; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Check]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Project: WISE; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Check]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. In Formulation: Project: Ares I; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Project: GPM; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Project: LDCM; Technology maturity, Critical technology maturity: [Check]; Technology maturity, Complexity of heritage technology: [Empty]; Design stability: [Empty]; Contractor performance: [Empty]; Development partner performance: [Check]; Funding issues: [Empty]; Launch manifest: [Empty]. Project: Orion; Technology maturity, Critical technology maturity: [Empty]; Technology maturity, Complexity of heritage technology: [Check]; Design stability: [Empty]; Contractor performance: [Check]; Development partner performance: [Empty]; Funding issues: [Check]; Launch manifest: [Empty]. Source: GAO analysis of NASA project data. [End of table] Technology Maturity was by far the most prevalent challenge, affecting 15 of the 19 projects. When combined with design instability—another metric related to technical difficulty-17 projects were affected. A discussion of each challenge follows. Technology Maturity: Our past work on systems acquisition has shown that beginning an acquisition program before requirements and available resources are matched can result in a product that fails to perform as expected, costs more, or takes longer to develop. We have found that these problems are largely rooted in the failure to match customer's needs with the developer's resources—technical knowledge, timing, and funding—when starting product development. In other words, commitments were made to deliver capability without knowing whether the technologies needed could really work as intended. Time and costs were consistently underestimated, and problems that surfaced early cascaded throughout development and magnified the risks facing the program. Our best practices work has shown that a technology readiness level (TRL) of 6— demonstrating a technology as a fully integrated prototype in a relevant environment—is the level of maturity needed to minimize risks for space systems entering product development.[Footnote 18] NASAs acquisition policy states that a TRL of 6 is desirable prior to integrating a new technology.[Footnote 19] Technology maturity is a fundamental element of a sound business case, and the absence is a marker for subsequent problems, especially in design.[Footnote 20] Similarly, our work has shown that the use of heritage technology— proven components that are being modified to meet new requirements—can also cause problems when the items are not sufficiently matured to meet form, fit, and function standards by the preliminary design review (PDR).[Footnote 21] NASA states in its Systems Engineering Handbook that particular attention must be given to heritage systems because they are often used in architectures and environments different from those in which they were designed to operate. Although NASA distinguishes critical technologies from heritage technologies, our best practices work has found critical technologies to be those that are required for the project to successfully meet customer requirements, regardless of whether or not they are based on existing or heritage technology. Therefore, whether technologies are labeled as "critical" or "heritage," if they are important to the development of the spacecraft or instrument—enabling it to move forward in the development process—they should be matured by PDR. Of the 14 projects for which we received data and that had entered the implementation phase, four entered this phase without first maturing all their critical technologies, and 10 encountered challenges in integrating or modifying heritage technologies. Additionally, two projects in formulation—Ares I and Orion—also encountered challenges with critical or heritage technologies. These projects did not build in the necessary resources for technology modification. For instance, the recent cost and schedule growth in the Mars Science Laboratory (MSL) highlights the problems that can be realized when a project proceeds past the formulation phase with immature technologies. MSL reported seven critical technologies were not mature at the time of its preliminary design review, and over a year later two of these technologies were still immature at the critical design review; however, the project moved forward into the implementation phase with established cost and schedule baselines and the lack of technology maturity contributed to an unstable design In part as a result of immature technologies and an unstable design, MSL delayed its launch date by 25 months, and development costs have grown by more than $660 million. In November 2008, the GRAIL project also moved beyond its PDR with an immature heritage technology—the reaction wheel assembly. This technology has been flown on other NASA missions, but the project team must modify it for GRAIL by integrating electronics into the assembly. NASA acknowledges in its Systems Engineering Handbook that modification of heritage systems is a frequently overlooked area in technology development and that there is a tendency on the part of project management to overestimate the maturity and applicability of heritage technology. NASA recognizes that as a result of not placing enough emphasis on the development of heritage technologies, key steps in the development process are not given appropriate attention, and critical aspects of systems engineering are overlooked. Design Stability: The importance of establishing a stable design at a project's critical design review (CDR) is also critical. The CDR provides assurance that the design is mature and will meet performance requirements. An unstable design can result in costly re-engineering and re-work efforts, design changes, and schedule slippage. Quantitative measures employed at CDR, such as percentage of engineering drawings, can provide evidence that the design is stable and "freeze" it to minimize changes in the future. Our work has shown that release of at least 90 percent of engineering drawings at the CDR provides evidence that the design is stable. Though NASA's acquisition policy does not specify how a project should achieve design stability by CDR, NASA's Systems Engineering Handbook adheres to this metric of 90 percent drawings released by the CDR. Eight projects in our assessment have already held their CDR and were able to provide us with the number of engineering drawings completed and released. None of these 8 projects met the 90 percent standard for design stability at CDR; however, NASA believes that some of these projects had stable designs and pointed to other activities that occurred prior to CDR as evidence. Nevertheless, the percentage of engineering drawings released at CDR by these 8 projects averaged less than 40 percent, and more than three-fourths of these projects had significant cost and/or schedule growth from their established baselines[Footnote 22] after their CDR when their design was supposed to be stable. Although all the cost and schedule growth for these projects cannot be directly attributed to a lack of design stability, we believe that this was a contributing factor. Discussions with project officials showed the metric was used inconsistently to gauge design stability. For example, Goddard Space Flight Center requires greater than 80 percent drawings released at CDR, yet we were told by several project officials that the rule of thumb for NASA projects is between 70 and 90 percent drawings released at CDR. However, there was no consensus among the officials. For example, one project manager from Goddard Space Flight Center told us the project is planning to have 70 percent of the drawings released at CDR; a project manager from the Jet Propulsion Laboratory cited that having 85 to 90 percent of the drawings released is what he prefers at CDR; otherwise, he does not consider the project design to be complete. Goddard's Chief Engineer said that, as a member of a design review board, he will generally question projects that have less than 95 percent of engineering drawings released, especially if the project is using heritage technologies. Officials added that at CDR it is more important to have drawings completed that relate to critical technologies than those related to integration activities. In addition to released drawings, NASA often relies on subject matter experts in the design review process and other methods to assure that a project has a stable design Some projects indicated that completing engineering models, which are preproduction prototypes, and holding sub-system level CDR's for instruments and components helped to assess design stability, at least in part. Officials for these projects indicated that use of engineering models helps decrease risk of flight unit development; projects that did not use engineering models indicated they might have caught problems earlier had they used them. For example, at CDR the Mars Science Laboratory's engineering models were incomplete and could have been a cause of concern. Mars Science Laboratory project officials were aware that avionics were an issue at CDR, but were unaware of future problems with other project components, such as the actuators. Project officials told us that if the engineering models for all subsystems had been completed at CDR, many of the later problems would have been caught and mitigated earlier in the process, thereby avoiding schedule delays. However, these project officials added that engineering models are expensive to employ and not all projects have the available funding required to utilize them. Contractor Performance: NASA relies heavily on the work of its contractors. Officials at five of the projects we reviewed indicated that the contractors for their projects had trouble moving their work forward after experiencing technical and design problems with hardware that disrupted development progress. Since about 85 percent of NASA's annual budget is spent by its contractors, the performance of these contractors is instrumental to the success of the projects. Shifts in the industrial base and a lack of expertise at the contractors affected performance. For example, project officials for the SOFIA project reported that the contractor for the aircraft modification was bought and sold several times during the development process. Project officials further reported that the contractor had limited experience with this type of work and did not fully understand the statement of work. Consequently, the contractor had difficulty completing this work, which led to significant cost overruns. While project officials told us that issues with that contractor have since been resolved, this year another SOFIA contractor that is responsible for developing hardware and software has performed poorly, which officials attribute to a recent buyout of the company. In addition, agency officials said that NASA is a low priority for the contractor, and the project is finding it difficult to exert pressure to ensure better performance. Project officials told us that they currently have three people at the contractor's site as a permanent presence. They added that if the contract were to be canceled due to poor performance, this work would be brought in-house and would result in a one year delay. In addition, the Glory project has struggled for several years to develop a key instrument. The Glory project manager cited management inefficiencies with the instrument's contractor including senior leadership changes, a loss of core competencies because of a plant closure, and a lack of proper decision authority. The contractor agreed that the plant closure and the need to re-staff were major project challenges. Development Partner Performance: Six projects in our review encountered challenges with their development partners. In these cases the development partner could not meet their commitments to the project within the planned schedules. For example, NASA collaborated with the European Space Agency (ESA) on the Herschel space observatory. NASA delivered its two instruments to ESA in a timely manner, but ESA encountered difficulties developing its instruments, and the result was a 14-month delay in Herschel's schedule. Because of this delay, NASA incurred an estimated $39 million in cost growth because of the need to fund component developers for a longer period of time than originally planned. We found that of the projects that are currently in implementation and have experienced cost and/or schedule growth, those with international or domestic partners experienced more than one-and-a-half times as much schedule growth on average as those with no partner. Table 3 below shows the average schedule growth for projects with partners as compared to those without partners. Table 3: Schedule Growth for Projects with and without Partners: Project with partners: Aquarius; Schedule growth (in months): 10. Project with partners: Herschel; Schedule growth (in months): 21. Project with partners: LRO; Schedule growth (in months): 8. Project with partners: MSL; Schedule growth (in months): 25. Project with partners: NPP; Schedule growth (in months): 33. Project with partners: SOFIA; Schedule growth (in months): 12. Project with partners: Average schedule growth; Schedule growth (in months): 18. Project without partners: Glory; Schedule growth (in months): 16. Project without partners: Kepler; Schedule growth (in months): 9. Project without partners: SDO; Schedule growth (in months): 18. Project without partners: WISE; Schedule growth (in months): 1. Project without partners: Average schedule growth; Schedule growth (in months): 11. Source: GAO Analysis of NASA data. [End of table] Funding Issues: During the course of our review, we identified six projects in the implementation phase, as well as three projects still in formulation, that had experienced issues related to the project's funding because of issues such as agency-directed funding cuts early in the project life-cycle and projects whose budgets do not match the work expected to be accomplished. For example, NASA management cut $35 million from the Kepler project's fiscal year 2005 budget—a cut amounting to one- half of the project's budget for the year. Contractor officials told us that this forced the shutdown of significant work, interrupted the overall flow and scheduling for staff and production, and required a renegotiation of contracts. This funding instability, according to a NASA project official, contributed to an overall 20-month delay in the project's schedule and about $169 million in cost growth. The funding instability for Kepler affected more than that one project. The WISE project had to extend the formulation phase since funding was unavailable at the time of the confirmation review in November 2005. According to NASA and contractor officials, the WISE project experienced funding cuts when NASA took money from that project to offset increased costs for the Kepler project. As a result of the extended formulation phase, the WISE project manager told us that development costs increased and the launch readiness date slipped 11 months. This is an example of how, when problems arise, one project can become the bill-payer for another project, making it difficult to manage the portfolio and make investment decisions. We also identified several projects where, according to NASA officials, the projected budget was inadequate to perform work in certain fiscal years. For example, the Constellation program's poorly phased funding plan has diminished both the Ares I and Orion projects' ability to deal with technical challenges. NASA initiated the Constellation program relying on the accumulation of a large rolling budget reserve in fiscal years 2006 and 2007 to fund program activities in fiscal years 2008 through 2010. Thereafter, NASA anticipated that the retirement of the space shuttle program in 2010 would free funding for the Constellation program. The program's risk management system identified this strategy as high risk, warning that shortfalls could occur in fiscal years 2009 through 2012. According to the Constellation program manager, the program's current funding shortfalls have reduced the flexibility to resolve technical challenges. In addition, the James Webb Space Telescope project had to delay its scheduled launch date by one year in part because of poor phasing of the project's funding plan. Launch Manifest Issues: We identified four projects in our assessment that are experiencing launch delays or other launch manifest-related challenges. By their nature, launch delays can contribute significantly to cost and schedule growth, as months of delay can translate into millions of dollars in cost increases. For example, the Solar Dynamics Observatory (SDO) project missed its scheduled launch date in August 2008 because of test scheduling and spacecraft parts problems. This delay resulted in the SDO project moving to the end of the manifest for the Atlas V launch vehicles on the East coast, causing an 18-month launch delay and $50 million cost increase. While the primary reason for the cost growth is that the SDO project could not meet its original schedule for launch, the project is incurring additional costs to maintain project staff longer than originally planned as they await their turn in the launch queue. According to SDO officials, this has also affected staffing at Goddard Space Flight Center since these personnel were scheduled to move to other projects. Furthermore, launch delays of one project can potentially impact the launch manifest for other projects. The 25-month delay of the Mars Science Laboratory project has the potential to cause disruptions for other projects on the launch manifest in late 2011, including those outside of NASA, since planetary missions—those missions that must launch in a certain window because of planetary alignments—receive launch priority to take advantage of optimal launch windows. Some NASA projects are also experiencing launch manifest-related challenges. For example, the Gravity Recovery and Interior Laboratory project is monitoring the availability of trained launch personnel as that mission is the last to launch on the Delta II vehicle. United Launch Alliance[Footnote 23] officials told us that they are taking active steps, such as cross-utilizing the Delta II personnel with other launch vehicles, to ensure that trained launch personnel are available for all the remaining Delta II launches. In addition, the recent failure of the Taurus XL launch vehicle during the launch of the Orbiting Carbon Observatory has the potential to delay the Glory mission if the Taurus XL is not cleared for use before Glory has corrected its technical problems. Project Assessments: The 2-page assessments of the projects we reviewed provide a profile of each project and describe the challenges we identified. On the first page, the project profile presents a general description of the mission objectives for each of the projects; a picture of the spacecraft or aircraft; a schedule timeline identifying key dates for the project; a table identifying programmatic and launch information; and a table showing the baseline year cost and schedule estimates and the most current available cost and schedule data; a table showing the challenges relevant to the project; and a project status narrative. On the second page of the assessment, we provide an analysis of the project challenges and the extent to which each project faces cost, schedule, or performance risk because of these challenges. In addition, NASA project offices were provided an opportunity to review drafts of the assessments prior to their inclusion in the final product, and the projects provided both technical corrections and more general comments. We integrated the technical corrections as appropriate and characterized the general comments below the detailed project discussion. See figure 2 below for an illustration of the layout of each two-page assessment. Figure 2: Illustration of Project Two-Page Summary: [Refer to PDF for image: illustration] A. General description of mission’s science objectives. B. Illustration of spacecraft or aircraft. C. Schedule timeline identifying key dates for the project including when the project began formulation, major design reviews, confirmation to begin the implementation phase, and scheduled launch readiness. D. (Project Essentials) Programmatic information including the responsible NASA center, international or domestic partners, major contractors, and launch information E. (Project Performance) Cost and schedule baseline estimates and the latest estimate updates. F. (Project Challenges) Summary listing the challenges facing the project based on a successful acquisition business case. G. (Project Status) Brief narrative describing current status of the project with regard to the challenges identified. H. (Detailed Project Discussion) Analysis of project challenges and the extent to which each project faces cost, schedule, or performance risk due to these challenges. I. (Project Office Comments) General comments provided by the cognizant project office. Source: GAO. Common Name: Aquarius: Aquarius: Figure: Illustration of Aquarius: Source: Aquarius Project Office (artist depiction). [End of figure] Aquarius is a satellite mission developed by NASA and the Space Agency of Argentina (Comision Nacional de Actividades Espaciales, CONAE) to investigate the links between the global water cycle, ocean circulation, and the climate. It will measure global sea surface salinity. The Aquarius science goals are to observe and model the processes that relate salinity variations to climatic changes in the global cycling of water and to understand how these variations influence the general ocean circulation. By measuring salinity globally for 3 years, Aquarius will provide an unprecedented new view of the ocean's role in climate. Formulation: Formulation start: 12/03; Preliminary design review: 6/05; Project confirmation: 9/05; Critical design review: 9/06. Implementation: GAO review: 12/09; Launch readiness date: 5/10. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partner: Argentina's National Committee of Space Activities (CONAE). Major Contractors: in-house development. Projected Launch Date: May 23, 2010. Launch Location: Vandenberg AFB, Calif. Launch Vehicle: Delta II. Mission Duration: 3 years for Aquarius mission; 5 years for SAC-D (CONAE) mission. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2008): $241.8; Latest (Oct. 2009): $260.0; Change: 7.5%. Formulation Cost: Baseline Est. (FY 2008): $35.5; Latest (Oct. 2009): $35.6; Change: 0.3%. Development Cost: Baseline Est. (FY 2008): $192.7; Latest (Oct. 2009): $208.6; Change: 8.3%. Operations Cost: Baseline Est. (FY 2008): $13.6; Latest (Oct. 2009): $15.8; Change: 16.2%. Launch Schedule: Baseline Est. (FY 2008): 7/2009; Latest (Oct. 2009): 5/2010; Change: 10 months. Project Challenges: * Design Stability; * Development Partner Performance; * Funding Issues. Project Status: The launch of Aquarius has been delayed from July 2009 to May 2010 because of delays in CONAE's spacecraft development. The launch delay, which added costs to the project, prompted NASA to report to the Congress that the Aquarius project exceeded its development schedule baseline. NASA completed its development of the Aquarius instrument, which is currently awaiting integration with the Argentine-developed spacecraft. Project officials are concerned about potential future delays during testing and integration. The project received $15.6 million under the American Recovery and Reinvestment Act of 2009 that will maintain the current Aquarius workforce through launch. Aquarius: Detailed Project Discussion: The only critical technology the project office identified was the Aquarius instrument itself, which includes the scatterometer and the radiometer components. The project deemed the instrument mature at the preliminary design review because the instrument uses heritage technologies, even though those technologies were brought together in a different form, fit, and function for use on Aquarius. The instrument design, however, was not stable at the critical design review (CDR) as the Aquarius project had released only 16 percent of the engineering drawings. Project officials told us that some detailed parts were either not accounted for or not very mature at CDR, and they needed a follow-on review to clear up the issue. For example, details in the design of a connector arm of a reflector to the instrument were lagging. Project officials added that the Aquarius instrument design was far ahead of development of CONAE's spacecraft, so the project could not finalize and release all the instrument drawings until CONAE completed the spacecraft design. To help minimize project risk in the interim, project officials said NASA provided CONAE with a structural model to work with as the Argentines developed the spacecraft. However, the Aquarius project did not build or test engineering models because of the funding constraints. Instead, breadboard units of most components were assembled to check the designs and interfaces. The development of Aquarius only involved the actual flight units, a type of project referred to by NASA as a "protoflight." The completed Aquarius instrument arrived in Argentina on June 3, 2009. The instrument is currently awaiting integration with the spacecraft bus, which cannot take place until CONAE corrects problems with identified non-compliances. Specifically, there is electrical interference from CONAE instruments that may affect Aquarius' performance. These non-compliances will need to be corrected before the CONAE's instruments are integrated with the spacecraft bus. The CONAE's instruments are currently being tested. After those efforts are completed, the project will integrate the Aquarius instrument with the spacecraft bus. Aquarius' schedule has slipped 10 months, prompting NASA to report, as required by law, to the Congress in 2008 that the Aquarius project had exceeded its development schedule baseline by more than 6 months. According to project officials and budget documents, a delay in development of the spacecraft bus by CONAE was the primary reason for the schedule slip. There have been additional delays. CONAE has recently experienced up to 16 weeks of schedule slip because of delayed delivery of its ROSA instrument and long-lead components such as s-band transponders and GPS receivers. These delays in turn are causing a challenge with integration/test planning and a decrease in schedule reserves that could ultimately delay the current May 22, 2010 launch readiness date. At this time, NASA has personnel at CONAE to assess and monitor the integration activities and correction of non- compliances. Since no funds are being exchanged between the U.S. and Argentina for this project, NASA bears the costs it incurs associated with any further delays. Project officials indicated that past schedule slips increased NASA's cost by $10.7 million. They also noted that this cost increase does not include increased launch vehicle costs because of the delay or Delta-II launch site maintenance costs at Vandenberg Air Force Base. The project received $15.6 million under the American Recovery and Reinvestment Act of 2009 that will maintain the current Aquarius workforce through launch. Project Office Comments: The Aquarius project provided technical comments to a draft of this assessment, which were incorporated as appropriate. The project officials also commented that they concur with GAO's assessment with the exception that they believe the instrument design was stable at critical design review. They commented that although only 16 percent of drawings were released at that review, over 80 percent of the drawings were completed and the Standing Review Board indicated that design was stable and mature enough for Aquarius to proceed with development. [End of Aquarius Assessment] Common Name: CLV: Ares I Crew Launch Vehicle (CLV): Figure: Illustration of Ares I Crew Launch Vehicle (CLV): Source: Ares Project Office (artist depiction). [End of figure] NASA's Ares I Crew Launch Vehicle, as part of the Constellation program, will carry the Orion Crew Exploration Vehicle into low Earth orbit for missions to the International Space Station and the Moon. The mission of the Ares I project is to deliver a safe, reliable, and affordable launch system with a 24.5-metric ton lift capability. Formulation: Formulation start: 9/05; Preliminary design review: 9/08; GAO review: 12/09; Project confirmation: 9/05. Implementation: Critical design review: 9/11; Launch readiness date: 3/15. Project Essentials: NASA Center Lead: Marshall Space Flight Center; Partners: None; Major Contractors: Alliant Techsystems, Pratt and Whitney Rocketdyne, Boeing; Projected Launch Date: March 2015; Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Ares I; Mission Duration: N/A; Project Performance (then year dollars in millions): Preliminary Estimate of Project Life Cycle Cost*; Latest (Oct. 2009): $17,000 to $20,000. *This estimate is preliminary, as the project is in formulation and there is still uncertainty in the value as design options are explored. NASA uses these estimates for planning purposes. This estimate is for the Ares I vehicle only. Launch Schedule: 3/2015. Project Challenges: * Complexity of Heritage Technology; * Funding Issues. Project Status: Technical and design challenges within the Ares I project are proving difficult, costly, and time-intensive to resolve. As a result of technical challenges and design modifications, the cost of Ares I developmental contracts has increased over $500 million since 2007. The Ares project received nearly $109 million under the American Recovery and Reinvestment Act of 2009 that will be used to manufacture and assemble engine components for development testing. The first manned launch has slipped from September 2014 to March 2015 to accommodate these challenges. Ares I Crew Launch Vehicle (CLV): Detailed Project Discussion. Citing the use of heritage systems and existing technologies in the design of the Ares I, project officials did not identify any critical technologies. We found, however, that technical and design challenges within the Ares I project are proving difficult, costly, and time- intensive to resolve. For example, NASA has identified thrust oscillation as a technical issue. Thrust oscillation, which causes shaking during launch and ascent, occurs in some form in every solid rocket engine. Computer modeling indicates that there is a possibility that the magnitude and frequency of thrust oscillation within the first stage may be outside the limits of the Ares I design and could cause excessive vibration in the Orion capsule and threaten crew safety. The Ares I project is pursuing multiple design solutions, but does not expect to resolve the thrust oscillation issue until the Constellation program's preliminary design review, currently scheduled for March 2010. NASA is also concerned that vibroacoustics—the pressure of the acoustic waves produced by the firing of the Ares I first stage and the rocket's acceleration through the atmosphere—may cause unacceptable structural vibrations throughout Ares I and Orion and force NASA to qualify components to higher vibration tolerance thresholds than originally expected. Analysis of the Ares I flight path also indicates that, under some conditions, the Ares I vehicle could hit the launch tower during liftoff. NASA plans to deal with this issue by steering the Ares I launch vehicle away from the tower and not launching when southerly winds exceed 15 to 20 knots. Since the project is still in formulation, NASA has not released official cost and schedule estimates for the Ares I project. NASA officials stated that these estimates will be made available when the project moves into implementation, or at the conclusion of the Constellation Program's non-advocate review. However, the value of various development contracts for the Ares I have increased by $500 million since 2007, and the first manned launch has slipped from 2014 to 2015. The Constellation program's poorly phased funding plan has diminished the Ares I project's ability to deal with technical challenges. NASA initiated the Constellation program relying on the accumulation of a large rolling budget reserve in fiscal years 2006 and 2007 to fund program activities in fiscal years 2008, 2009, and 2010. Thereafter, NASA anticipated that the retirement of the space shuttle program in 2010 would free funding for the Constellation program. The program's integrated risk management system identified this strategy as a high risk and warned that funding shortfalls could occur in fiscal years 2009 through 2012. According to the Constellation program manager, the program's current funding shortfalls are reducing the flexibility to resolve technical challenges. The Ares project received nearly $109 million under the American Recovery and Reinvestment Act of 2009 that will be used to manufacture and assemble engine components for development testing, completion of a test stand, and preparation for test operations. Nevertheless, in September 2009, an independent commission formed by the President to study the future of U.S. human spaceflight reported that NASA's plans for the Constellation program to return man to the moon by 2020 are unexecutable without drastic increases to NASA's current budget profile. Project Office Comments: The Ares I project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that they believe the project is successfully progressing in the design of a new crew launch vehicle and meeting all major technical and programmatic milestones. The officials added that the project continues to use proven risk management systems to identify technical risk and is using proven engineering and management techniques to effectively mitigate these risks while limiting cost and schedule impacts to the overall program. [End of Ares I Crew Launch Vehicle (CLV) Assessment] Common Name: Glory: Glory: Figure: Illustration of Glory. Source: Glory Project Office (artist depiction). [End of figure] The Glory project is a low-Earth orbit satellite that will contribute to the U.S. Climate Change Science Program. The satellite has two principal science objectives: (1) collect data on the properties of aerosols and black carbon in the Earth's atmosphere and climate systems and (2) collect data on solar irradiance. The satellite has two main instruments —the Aerosol Polarimetry Sensor (APS) and the Total Irradiance Monitor (11M)—as well as two cloud cameras. The TIM will allow NASA to have uninterrupted solar irradiance data by bridging the gap between NASA's Solar Radiation and Climate Experiment and the National Polar Orbiting Environmental Satellite System (NPOESS) missions. Formulation: Formulation start: 9/05; Preliminary design review: 9/08; Project confirmation: 12/05; Critical design review: 7/06. Implementation: GAO review: 12/09; Launch readiness date: 10/10. Project Essentials: NASA Center Lead: Goddard Space Flight Center; International Partner: None; Major Contractors: Raytheon Space and Airborne Systems, University of Colorado Laboratory for Atmospheric and Space Physics, Orbital Sciences Corporation; Projected Launch Date: October 2010; Launch Location: Vandenberg AFB, Calif. Launch Vehicle: Taurus XL; Mission Duration: 3 years (5 year goal). Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $347.9; Latest (Oct. 2009): $394.9; Change: 13.5%. Formulation Cost: Baseline Est. (FY 2009): $70.5; Latest (Oct. 2009): $70.8; Change: 0.0%. Development Cost*: Baseline Est. (FY 2009): $259.1; Latest (Oct. 2009): $296.1; Change: 14.3%. Operations Cost: Baseline Est. (FY 2009): $18.3; Latest (Oct. 2009): $28.1; Change: 53.6%. Launch Schedule: Baseline Est. (FY 2009): 6/2009; Latest (Oct. 2009): 10/2010; Change: 16 months. *Represents a 75% growth in development costs since the original baseline of $168.9 established in fiscal year 2008. Project Challenges: * Technology Maturity; * Complexity of Heritage Technology; * Design Stability; * Contractor Performance; * Launch Manifest. Project Status: Since Congress reauthorized the Glory project in fiscal year 2009, and a new cost and schedule baseline was established, costs and schedule have continued to increase. Furthermore, the project may experience additional delays because of uncertainty of the status of the Taurus XL launch vehicle, which failed to deliver a payload to orbit during a recent launch. Glory, now currently scheduled to launch not earlier than October 2010, has exceeded its new schedule baseline and will likely exceed its new cost baseline. Glory received $21 million under the American Recovery and Reinvestment Act of 2009 that will help offset increasing development costs. Glory: Detailed Project Discussion: The Glory project has experienced significant delays because of a technical problem; according to the project manager, a crack in the Single Board Computer (SBC) Printed Wiring Board was confirmed in January 2009. While the project was working through manufacturing issues with the SBC vendor, the project concurrently pursued an alternate solution to the problem using an SBC manufactured by a second vendor. In July 2009, project officials changed the baseline to the alternate SBC. The project will experience significant delays based on this decision since the new SBC will not be delivered until spring 2010. The project will need to use an engineering model of the SBC in the spacecraft for thermal vacuum testing until the new computer box is available in order to minimize schedule impact caused by the change. The Aerosol Polarimetry Sensor (APS) was delivered on March 9, 2009— over one year behind schedule and $103 million more than estimates at project confirmation—and was successfully integrated with the spacecraft in April 2009. This development of the APS, which is based on heritage technology, has resulted in significant cost increases and delays to the project. The instrument was the project's only immature technology at the mission preliminary design review in September 2005 and was beset by contractor performance issues throughout development. Project officials said that the APS development issues were not technical issues, but instead resulted from the contractor's inability to plan and execute the work and the closure and move of the contractor's facility. According to contractor officials, moving the APS development effort from one facility to another and deciding to finish building the instrument rather than doing a complete design analysis led directly to cost and schedule increases. The project's design was not stable at the mission critical design review as the project had released only 68 percent of its drawings. As of GAO's review, 99 percent of total drawings had been released. However, Glory's drawing count increased by 27 percent after the critical design review. This increase is attributed to the modification of drawings for heritage parts for Glory's unique configuration. As required by law, Glory had to be reauthorized by the Congress in fiscal year 2009 in order for the project to continue after a 53 percent increase in development cost, and a new cost and schedule baseline was established. Since that time there has been a 14 percent increase in development costs and a 10-month delay in the scheduled launch date. In total, since the original fiscal year 2008 baseline, the project's development costs have grown by 75 percent. Glory received $21 million under the American Recovery and Reinvestment Act of 2009 which will maintain the current contractor workforce through the launch. Recent failure of the Taurus XL launch vehicle during the launch of the Orbiting Carbon Observatory (OCO) further threatens to delay Glory's launch date. The launch failure Mishap Investigation Board (MIB) subsequently released findings and suggested corrective actions. Specifically, the MIB found that a payload fairing—a clamshell-shaped cover that encloses and protects a payload during early flight—failed to separate during ascent. Glory project officials indicated that Orbital Launch Systems Group will modify the fairing deployment design. Project Office Comments: The project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that all mission activities are on track to support the launch of Glory in 2010. However, project officials noted that following the February 2009 launch failure of the OCO, the Taurus XL launch vehicle requires approval for re-flight They added that the return-to-flight requirements continue to be addressed but the timeline for their completion remains a threat to Glory's launch date. [End of Glory Assessment] Common Name: GPM: Global Precipitation Measurement (GPM) Mission: Figure: Illustration of Global Precipitation Measurement (GPM) Mission: Source: GPM Project Office (artist depiction). [End of figure] The Global Precipitation Measurement (GPM) mission, a joint NASA and Japan Aerospace Exploration Agency (JAXA) project, seeks to improve the scientific understanding of the global water cycle and the accuracy of precipitation forecasts. The GPM is composed of a core spacecraft carrying two main instruments: a Dual-frequency Precipitation Radar (DPR) and a GPM Microwave Imager (GMI). In addition, the GPM project includes a second Low-Inclination spacecraft with a second GMI instrument. The GPM builds on the work of the Tropical Rainfall Measuring Mission and will provide the first opportunity to calibrate measurements of global precipitation. Formulation: Formulation start: 7/02; Preliminary design review: 11/08; Project confirmation: 11/09; Implementation: Critical design review: 12/09; GAO review: 12/09; Launch core spacecraft: 7/13; Launch low-inclination spacecraft: 11/14. Project Essentials: NASA Center: Goddard Space Flight Center; International Partner: Japanese Aerospace Exploration Agency (JAXA). Major Contractors: Ball Aerospace. Projected Launch Date: July 21, 2013. Launch Location: Tanegashima Island, Japan. Launch Vehicle: H-IIA (Japan). Mission Duration: 3 years (5 years consumables). Project Performance (then year dollars in millions): Preliminary Estimate of Project Life Cycle Cost*: Latest (Oct. 2009): not provided. *NASA suggested it will provide baseline estimates for this project when it proceeds from formulation into development. Launch Schedule: 7/2013. Project Challenges: * Technology Maturity; * Funding Issues. Project Status: Technical challenges surrounding the ability of the GPM spacecraft to burn up as it re-enters the atmosphere and minimize debris—-its demisability—-have delayed the establishment of cost and schedule baselines by 8 months. NASA has identified fiscal year 2009 as a high- risk year because of previous reductions in funding levels and low contingency reserves. GPM is slated to receive $32 million under the American Recovery and Reinvestment Act of 2009 with which NASA intends to accelerate construction of the GPM Microwave Imager (GMI) instrument for the core spacecraft to ensure successful launch of the mission at the earliest possible opportunity. Global Precipitation Measurement Mission (GPM): Detailed Project Discussion: The GPM spacecraft was designed to be demiseable–-that is it will burn up during re-entry into the Earth's atmosphere to limit orbital debris. However, in December 2008, a structural analysis at Johnson Space Center of GPM indicated that the spacecraft would not be demiseable as originally predicted by the GPM project office, which based its analyses on demisability of a similar predecessor spacecraft. The project delayed the start of the implementation phase and establishment of GPM cost and schedule baselines by 8 months because of the budgetary impact of the demisability issue. But the project continues to develop alternative design solutions in order to minimize the debris area as the spacecraft returns through the atmosphere during reentry. Specifically, the project manager said they will continue development of a fully demiseable aluminum propulsion tank as opposed to using a heritage-technology titanium propulsion tank, which would not fully burn up in the atmosphere upon re-entry. While the titanium propulsion tank would have been "off the shelf," the spacecraft would have had to be redesigned to accommodate it; whereas, the aluminum propulsion tank has already been specifically designed for GPM and has a larger propellant capacity, thus increasing mission capability and the possibility of post-prime mission operations. Project officials confirmed that the only critical technology they are maturing is the treatment processes for the propellant management device with the aluminum composite propulsion tanks. However, this technology was immature at the preliminary design review in November 2008. GPM technologies are largely heritage technologies patterned after those used on other NASA missions. According to project officials, the two main instruments—the JAXA-supplied Dual-frequency Precipitation Radar (DPR) and the NASA-supplied GPM Microwave Imager (GMI)—are based on heritage technology and therefore are not considered critical technologies. However, the DPR and GMI will have to be adapted to the GPM spacecraft design for this mission. In addition, NASA recently renegotiated the contract for GMI to account for delaying the delivery of the instrument from May 2009 to April 2011 because of past year's budget reductions. The GPM project has not reached a design review where we could assess design stability. The project currently has released 48 percent of its engineering drawings and anticipates releasing approximately 70 percent of its drawings prior to the mission critical design review. The project manager said that all design drawings are not needed at the time of the critical design review, especially assembly and integration drawings, which can be released after the design review. NASA had identified fiscal year 2009 as a high-risk year because of reduced funding levels and low contingency reserves. However, the GPM project is slated to receive $32 million under the American Recovery and Reinvestment Act of 2009. NASA intends to use these funds to accelerate construction of the GPM Microwave Imager (GMI) instrument for the core spacecraft to ensure successful launch of the mission at the earliest possible opportunity. Further, the project manager said GPM officials are discussing project requirements with NASA to make sure the budget contains sufficient reserves. While the project manager maintains that the prime mission requirements will be met, he said the project office may reduce requirements for the Low Inclination Observatory (LIO) mission, which is the second spacecraft under this project, due to launch in November 2014. Project Office Comments: The GPM project office provided technical comments on a draft of this assessment, which were incorporated as appropriate. Project officials also commented that the mission is on track for project confirmation in late 2009, and will then proceed into implementation. [End of Global Precipitation Measurement Mission (GPM) Assessment] Common Name: GRAIL: Gravity Recovery and Interior Laboratory (GRAIL): Figure: Illustration of GRAIL. Source: Courtesy of NASA/JPL-Caltech. [End of figure] The GRAIL mission will seek to determine the structure of the lunar interior from crust to core, advance our understanding of the thermal evolution of the Moon, and extend our knowledge gained from the Moon to other terrestrial-type planets. GRAIL will achieve its science objectives by placing twin spacecraft in a low altitude and nearly circular polar orbit. The two spacecraft will perform high-precision measurements between them. Analysis of changes in the spacecraft-to- spacecraft data caused by gravitational differences will provide direct and precise measurements of lunar gravity. GRAIL will ultimately provide a global, high-accuracy, high-resolution gravity map of the moon. Formulation: Formulation start: 12/07; Preliminary design review: 11/08; Project confirmation: 11/09; Implementation: Critical design review: 11/09; GAO review: 12/09; Launch readiness date 9/11. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partners: None. Major Contractors: Lockheed Martin. Projected Launch Date: September 8, 2011. Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Delta II Heavy. Mission Duration: 9 months. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $496.2; Latest (Oct. 2009): $496.2; Change: 0.0%. Formulation Cost: Baseline Est. (FY 2009): $50.5; Latest (Oct. 2009): $50.5; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $427.0; Latest (Oct. 2009): $427.0; Change: 0.0%. Operations Cost: Baseline Est. (FY 2009): $18.7; Latest (Oct. 2009): $18.7; Change: 0.0%. Launch Schedule: Baseline Est. (FY 2009): 9/2011; Latest (Oct. 2009): 9/2011; Change: 0 months. Project Challenges: * Complexity of Heritage Technology; * Launch Manifest. Project Status: In January 2009, the GRAIL project was confirmed and established its cost and schedule baselines. While the project relies heavily on heritage technologies, one technology, the reaction wheel assembly, was not mature at the preliminary design review. The development of the engineering model for this component was lagging and could affect the flight unit development. In addition, the project is concerned about availability of Delta II launch personnel since this mission is scheduled to be the last for this launch vehicle. There is also concern about GRAILs launch date since it is positioned very close to two planetary missions, which have launch priority. Gravity Recovery and Interior Laboratory (GRAIL): Detailed Project Discussion: The GRAIL project instruments are similar to those used in the Gravity Recovery and Climate Experiment (GRACE) mission, a project with similar twin satellites launched in March 2002 that made detailed measurements of Earth's gravity field. The project identified only one critical technology—the Time Transfer System in the Lunar Gravity Ranging System. The project currently has an engineering model of it and has deemed it mature. Project officials said they included no new technology in designing the GRAIL orbiters to keep the mission simple, cost effective, and as close to the GRACE mission as possible. Heritage technologies incorporated by the project include the Lunar Gravity Ranging System instrument, the MoonKam outreach cameras, Flight Software, and the reaction wheel assembly. The project had deemed all of these technologies as mature at the preliminary design review (PDR) except the reaction wheel assembly, which the project has identified as a risk to the project. Project officials told us that during formulation they reviewed the reaction wheel assembly and determined that it did not meet the standards for this mission, causing the project to undertake a new development effort. The officials added that the development of the engineering model for this component was lagging and could affect the flight unit schedule. In addition to the reaction wheel assembly, the project is tracking and managing several other risks. For example, project officials explained that mass reserve margin has been a priority for the project since PDR. They were concerned the project is still in development where mass can continue to grow as the design is matured. Project officials indicate that the dry mass is currently at 14 percent margin, above the 10 percent margin required by JPL standards as they near the critical design review. The project also identified a risk for long lead items and placed emphasis on obtaining quality parts, employing tiger teams to help mitigate the risk. At this time the project has received 85 percent of the items and has ordered additional commercial parts for those not yet obtained. In addition, project officials told us they are concerned about the availability of trained personnel to process the launch since GRAIL will be the last Delta II to launch. Project officials said that the project is also monitoring the launch schedule very closely since their launch date is currently between two planetary missions—one month after the Juno mission and one month prior to the new date for the Mars Science Laboratory (MSL). Planetary missions have specified windows for launch and are given priority for launch manifest scheduling. Consequently, changes to either Juno or MSL's launch date could impact GRAILs launch date. The project has not reached a design review where we could assess design stability. Although the project did not calculate the number of engineering drawings complete at PDR, project officials expect to have at least 98 percent of drawings released by the mission critical design review (CDR) in November 2009. This is primarily due to the high degree of heritage technology being utilized by the project. Project officials told us they plan to hold a series of component and sub-system design reviews prior to the mission CDR, and the project test program includes engineering model testing prior to the mission CDR. Project Office Comments: The GRAIL project provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials commented that they concur with GAO's assessment of the GRAIL project. [End of Gravity Recovery and Interior Laboratory (GRAIL) Assessment] Common Name: Herschel: Herschel: Figure: Illustration of Herschel: Source: ESA/AOES Medialab (artist depiction). [End of figure] The Herschel Space observatory, a collaborative project between NASA and the European Space Agency (ESA), will seek to discover how the first galaxies formed and how they evolved to give rise to present-day galaxies like our own. Herschel has the largest mirror ever built for a space telescope at 3.5 meters in diameter. The mirror will collect long-wavelength radiation from some of the coldest and most distant objects in the Universe. It will be able to observe dust-obscured and cold objects that are invisible to other telescopes. Additional targets for Herschel will include clouds of gas and dust where new stars are being born, disks out of which planets may form, and cometary atmospheres packed with complex organic molecules. Formulation: Formulation start: 1998; Preliminary design review: 7/00; Project confirmation: 3/01; Implementation: Critical design review: 7/01; Launch readiness date 3/09; GAO review: 12/09. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partner: European Space Agency (ESA). Major Contractors: in-house development. Launch Date: May 14, 2009. Launch Location: Kourou, French Guiana. Launch Vehicle: Ariane 5 (ESA Supplied). Mission Duration: 3 years (5 year goal). Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $325.4; Latest (Oct. 2009): $277.0; Change: -14.9%. Formulation Cost: Baseline Est. (FY 2009): $10.4; Latest (Oct. 2009): $10.4; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $117.0; Latest (Oct. 2009): $126.7; Change: 8.3%. Operations Cost: Baseline Est. (FY 2009): $198.0; Latest (Oct. 2009): $139.9; Change: -29.3%. Launch Schedule: Baseline Est. (FY 2009): 8/2007; Latest (Oct. 2009): 5/2009; Change: 21 months. Project Challenges: * Technology Maturity; * Design Stability * Development Partner Performance. Project Status: Herschel launched on May 14, 2009, after a 21-month delay. Since Herschel's 2007 baseline, ESA delayed Herschel's launch three times because of scope changes and challenges with integration of the instruments onto the spacecraft. These launch delays resulted in a project cost increase of $43 million and required NASA to report to the Congress that it exceeded its schedule baseline. Herschel: Detailed Project Discussion: Herschel launched on May 14, 2009, after a 21-month delay. After completion of the commissioning phase in July 2009, responsibility for operating the Herschel observatory transitioned from ESA's Space Operations Centre in Darmstadt, Germany, to the Herschel Science Centre in Madrid, Spain. Initial science observations began in October 2009, followed by routine science observations in November 2009. NASA developed and delivered components for two Herschel instruments— the Heterodyne Instrument for the Far Infrared (HIFI) and the Spectral and Photometric Imaging Receiver (SPIRE) instrument. However, during both the preliminary and critical design reviews, some of the critical technologies for these elements were considered immature. At the preliminary design review (PDR) for HIFI, five of the eight critical technologies were immature. Later, at critical design review (CDR), two of the eight HIFI critical technologies were still assessed as immature. SPIRE had a similar record. At the PDR for SPIRE, three of the five critical technologies were assessed as immature. Two years later at CDR, two of five SPIRE critical technologies were still assessed as immature. In addition to technology maturity issues, NASA committed to developing components for the HIFI and SPIRE instruments before achieving design stability for the instruments. At the CDR for both the HIFI and SPIRE instruments, NASA had released less than 10 percent of the engineering design drawings. According to the project officials, this was primarily because of the fact that ESA's interface drawings were preliminary. The officials also said that the lack of timeliness in the submission of design drawings is a challenge when the project has to depend on multiple partners for input. According to project officials, both the HIFI and SPIRE teams relied heavily on the use of engineering models to verify that adequate maturity of the designs was achieved at CDR, and used the model development to change the final design of the flight components. In addition, project officials indicated that the procurement of long lead-time items was a constant challenge during development. Herschel's $43 million growth in life cycle costs can be largely attributed to technical integration problems, which resulted in launch delays. According to NASA officials, ESA's contractor could not complete development of its instruments or integrate Herschel's instruments in a timely manner, prompting ESA to pull the integration work in-house. In addition, problems were found during subsystem testing of NASA's components in Europe. According to the project office, the HIFI failed during instrument integration and SPIRE had problems with the wiring that connects its detectors during the system thermal vacuum test. NASA faced some technical problems with development of components for the HIFI and SPIRE instruments, resulting in about $3.9 million of cost growth. Project officials said the remaining increase of about $39 million is because of the three slips in Herschel's launch date since the project's baseline was established in February 2007. This amount is attributed to the additional costs to the project of maintaining a workforce to support testing and integration activities. Based on a 14-month delay in launch date, as required by law, NASA reported to the Congress in February 2008 that the Herschel project would exceed its schedule baseline by more than 6 months. The project experienced a subsequent 7-month slip in its launch after further delays in spacecraft integration. Project Office Comments: The Herschel project provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that they believe the reason for having so few drawings released during the critical design review was mostly attributed to delays by the ESA Herschel project office firming up interface definitions, a delay that they say prevented releasing the final version of the drawings. [End of Herschel Assessment] Common Name: Juno: Juno: Figure: Illustration of Juno. Source: NASNJPL. [End of figure] The Juno mission seeks to improve our understanding of the origin and evolution of Jupiter. Juno plans to achieve its scientific objectives by using a simple, solar-powered spacecraft to make global maps of the gravity, magnetic fields, and atmospheric conditions of Jupiter from a unique elliptical orbit. The spacecraft carries precise, highly sensitive radiometers, magnetometers, and gravity science systems. Juno is slated to make 32 orbits to sample Jupiter's full range of latitudes and longitudes. From its polar perspective, Juno is designed to combine local and remote-sensing observations to explore the polar magnetosphere and determine what drives Jupiter's remarkable auroras. Formulation: Formulation start: 7/05; Preliminary design review: 5/08; Project confirmation: 8/08; Implementation: Critical design review: 4/09; GAO review: 12/09; Launch readiness date 8/11. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partners: Agencia Spaziale Italiana (ASI)- Selex Galileo; ASI - Thales Alenia Space; Centre Spatial de Liege (CSL) Belgian Science Policy; Centre National d'Etudes Spatiales (CNES) - Centre d'Etude. Major Contractors: Lockheed Martin. Projected Launch Date: August 5, 2011. Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Atlas V. Mission Duration: 6 Years. Project Performance (then year dollars in millions); Total Project Cost: Baseline Est. (FY 2009): $1,107.0; Latest (Oct. 2009): $1,107.0; Change: 0.0%. Formulation Cost: Baseline Est. (FY 2009): $186.3; Latest (Oct. 2009): $186.3; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $742.3; Latest (Oct. 2009): $742.3; Change: 0.0%. Operations Cost: Baseline Est. (FY 2009): $178.4; Latest (Oct. 2009): $178.4; Change: 0.0%. Launch Schedule: Baseline Est. (FY 2009): 8/2011; Latest (Oct. 2009): 8/2011; Change: 0 months. Project Challenges: * Complexity of Heritage Technology; * Design Stability * Development Partner Performance. Project Status: The Juno project recently established its cost and schedule baseline. One of the heritage technologies for Juno was reassessed as immature after the mission preliminary design review. In addition, two components remain on the critical path and could cause a delay to Juno's launch. An earthquake in central Italy in April 2009 caused damage to a factory in which a Juno component was being developed. Project officials and the Italian Space Agency are working to mitigate the project risks from this event. Juno: Detailed Project Discussion: The Juno project office indicated that there are no new critical technologies. The project did identify four heritage technologies that were all deemed mature at the preliminary design review in May 2008— the Stellar Reference Unit, Solar Cells, Toroidal Low Gain Antenna (TLGA), and Waves Instrument. However, after the preliminary design review, the project reassessed the TLGA as immature when it was determined that the materials being used in the highly charged particle environment could store an electrical charge, which would in turn interfere with some lower-level science requirements from two of the instruments on the spacecraft. The project plans to coat the surface of the TLGA with germanium to provide a discharge path to the grounded metal structure. In February 2009, the project also widened the Solar Array Panels to increase power, which in turn increased the mass, causing the project to exceed mass margin requirements. Project officials told us they performed an analysis of the spacecraft's mass and made modifications to achieve the standard 10 percent mass margin required by JPL standards at project critical design review (CDR). They added that this will continue to be monitored closely. The Juno project's design was unstable at the CDR as the project had released 77 percent of the engineering drawings. Project officials, however, said they used engineering models for all instruments to demonstrate design maturity at CDR. For some spacecraft components, the Juno project did not build or test engineering models because they were of heritage designs For example, some spacecraft components utilized are very similar to the ones used on the Mars Reconnaissance Orbiter such as some Command and Data Handling (C&DH) and power cards. Therefore, the project accepted some of the spacecraft card designs based on qualification testing. In addition, subsystem and component- level reviews were held prior to the mission CDR, and project officials told us the results of these lower-level reviews provided evidence that the design was stable. Juno project officials said they must have all instruments delivered by July 2010 in order to begin integration and testing on schedule. They identified two components that they believe are challenges for the Juno project to maintain its schedule—the Jovian Infrared Auroral Mapper (JIRAM) instrument and the C&DH module. JIRAM experienced delays early in design and manufacturing work, and has only 20 days of schedule margin remaining to meet the July 2010 deadline. The C&DH module delays are a result of late workforce ramp-up and start of the flight design effort. Project officials said that test and integration will begin with a test unit for the C&DH module and that they will incorporate the flight model when it is complete. Juno's international partner, the Italian Space Agency (ASI), has experience delays because of the April 2009 earthquake in central Italy. The factory where the Ka-band Translator was being developed was badly damaged and rendered unusable, causing ASI to develop a comprehensive plan to move that development to another factory. There is still schedule margin available by dual qualifying the engineering model as the possible flight model. The project will continue working toward a separate flight model, but has accepted the risk associated with using a flight-qualified engineering model instead if this becomes necessary. Project Office Comments: The Juno project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that the number of drawings released at the critical design review was per NASA's plan and do not represent design instability. They added that neither the JIRAM instrument nor the Ka- band Translator are needed to meet the science requirements, and Juno could launch without them if necessary. In addition, they commented that multiple schedule workarounds exist if the C&DH module deliveries are further delayed. [End of Juno Assessment] Common Name: JWST: James Webb Space Telescope (JWST): Figure: Illustration of JWST. Source: JWST Project Office (artist depiction). [End of figure] The James Webb Space Telescope (JWST) is a large, infrared-optimized space telescope that is designed to find the first galaxies that formed in the early universe. Its focus will include searching for first light, assembly of galaxies, origins of stars and planetary systems, and origins of the elements necessary for life. JWST's instruments will be designed to work primarily in the infrared range of the electromagnetic spectrum, with some capability in the visible range. JWST will have a large mirror, 6.5 meters (21.3 feet) in diameter and a sunshield the size of a tennis court. Both the mirror and sunshade will not fit onto the rocket fully open, so both will fold up and open once JWST is in outer space. JWST will reside in an orbit about 1.5 million kilometers (1 million miles) from the Earth. Formulation: Formulation start: 3/99; Preliminary design review: 3/08; Project confirmation: 7/08; Implementation: GAO review: 12/09; Critical design review: 3/10; Launch readiness date 6/14. Project Essentials: NASA Center Lead: Goddard Space Flight Center. International Partners: European Space Agency (ESA), Canadian Space Agency (CSA). Major Contractors: Northrop Grumman. Projected Launch Date: June 2014. Launch Location: Kourou, French Guiana. Launch Vehicle: Ariane 5 (ESA Supplied). Mission Duration: 5 years (10 year goal). Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $4,963.6; Latest (Oct. 2009): $4,963.6; Change: 0.0%. Formulation Cost: Baseline Est. (FY 2009): $1,800.1; Latest (Oct. 2009): $1,800.1; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $2,581.1; Latest (Oct. 2009): $2,581.1; Change: 0.0%. Operations Cost: Baseline Est. (FY 2009): $582.4; Latest (Oct. 2009): $582.4; Change: 0.0%. Launch Schedule: Baseline Est. (FY 2009): 6/2014; Latest (Oct. 2009): 6/2014; Change: none. Project Challenges: * Complexity of Heritage Technology; * Funding Issues. Project Status: After confirmation, JWST established a baseline life cycle cost of $4.96 billion and a June 2014 launch date. This constitutes about a half billion increase and a 1-year launch delay from NASA's fiscal year 2006 re-plan. Concerns over low contingency funding, project development, and remaining technical challenges were cited as reasons for delaying the launch. Additionally, the project received $75 million under the American Recovery and Reinvestment Act of 2009. NASA plans to use these funds for spacecraft and instrument development, in hopes of meeting its launch date. James Webb Space Telescope (JWST): Detailed Project Discussion: The JWST project identified 10 critical technologies and assessed all of them as mature during the preliminary design review in March 2008. However, the data from the project indicated that two of its 15 heritage technologies—the Solar Array and the S-band Transponder—are still immature over a year after the preliminary design review. Project officials indicated they are not tracking the development of these technologies as project risks, and one official said they are not overly concerned about maturation of these two heritage technologies. The Fine Sun Sensor Assembly (FSSA), which is also based on heritage technology, has not yet been selected, but the project intends to select a specific FSSA unit from among those that have previously been flown on other missions. In addition, the JWST project office has released 87 percent of its design drawings as of September 2009, and anticipates releasing 95 percent of its design drawings by the critical design review in March 2010. In 2009, we reported that the project had to address several issues related to testing identified at the project's preliminary design review. One concern was that the project planned to conduct only one test at the highest level of assembly in the cryogenic vacuum chamber at Johnson Space Center. The preliminary design review board advised the project to add another test cycle to its schedule. According to a project official, JWST still plans to conduct only one test campaign at the highest level of assembly, but the official added that cost and schedule reserves have been set aside to accommodate additional testing if needed. The review board was also concerned that the project was not planning to test the deployment of the sunshield at the highest level of assembly in the cryogenic chamber, and required the project to defend its current plans for sunshield deployment testing, including the possibility of additional tests. Project officials said they are studying this issue and hope to have an updated plan for sunshield testing by mission critical design review. In addition, the project heeded the review board's recommendation to add a center of curvature test on the Optical Telescope Element. The JWST project was re-planned in fiscal year 2006 after a $1 billion cost increase and a 2-year schedule delay on the project. In fiscal year 2009, the project established its baseline with a life cycle cost of $4.96 billion and a June 2014 launch date. This represents about a $500 million increase over NASA's 2006 replan figures and has resulted in another 1-year delay of the launch readiness date. According to the project manager, in July 2008, JWST adjusted its launch date from its previous June 2013 date to June 2014 in order to accommodate low budget and schedule reserves. Prior to this schedule delay, an independent review team expressed concern that budget and schedule reserves were too low to meet the June 2013 launch date. The revised June 2014 launch date was also based on assessments of the project's development progress to date, estimates of the technical challenges remaining, and the need to maintain an acceptable level of risk. JWST received $75 million under the American Recovery and Reinvestment Act of 2009 that it plans to use for spacecraft and instrument development activities including design and fabrication of key component systems. NASA believes these activities will increase the likelihood that JWST will launch on its planned launch date. Project Office Comments: The JWST project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that they do not consider the heritage technologies to be immature. [End of James Webb Space Telescope (JWST) Assessment] Common Name: Kepler: Kepler: Figure: Illustration of Kepler. Source: Kepler Project Office. [End of figure] The Kepler mission was designed to discover Earth-like planets in orbit around stars in our galaxy. The goal of the mission is to detect tens or even hundreds of Earth-size planets in the habitable zones of stars similar to our own sun. The habitable zone is the region around a star where the temperature of a terrestrial-type planet can be expected to allow water to exist in liquid form on the planet's surface, thereby increasing the probability of life. Kepler will explore the structure and diversity of planetary systems by conducting a census of extra-solar terrestrial planets using a photometer in heliocentric orbit to observe the dimming of starlight caused by planetary transits. Formulation: Formulation start: 10/01; Preliminary design review: 10/04; Project confirmation: 5/05; Implementation: Critical design review: 10/06; Launch readiness date 3/09; GAO review: 12/09. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partner: None. Major Contractors: Ball Aerospace and Technologies Corp. Launch Date: March 6, 2009. Launch Location: Cape Canaveral AFS, Fla. Launch Vehicle: Delta II. Mission Duration: 3.5 years. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $497.5; Latest (Oct. 2009): $604.6; Change: 21.5%. Formulation Cost: Baseline Est. (FY 2009): $138.1; Latest (Oct. 2009): $141.2; Change: 2.2%. Development Cost: Baseline Est. (FY 2009): $312.8; Latest (Oct. 2009): $390.3; Change: 24.8%. Operations Cost: Baseline Est. (FY 2009): $46.6; Latest (Oct. 2009): $73.1; Change: 56.9%. Launch Schedule: Baseline Est. (FY 2009): 6/2008; Latest (Oct. 2009): 3/2009; Change: 9 months. Project Challenges: * Complexity of Heritage Technology; * Contractor Performance; * Funding Issues. Project Status: Kepler successfully launched in March 2009 and is currently in operations. However, since being baselined in fiscal year 2007, Kepler's development costs have increased by about 25 percent and its schedule has increased by 9 months. Project officials attribute the cost and schedule growth to many things, including a $35 million budget reduction in fiscal year 2005. This funding instability contributed to an overall 20-month delay in the project's schedule and about $169 million in cost growth. NASA reported to the Congress that both Kepler's development costs and schedule exceeded its baselines. Kepler: Detailed Project Discussion: Kepler successfully launched in March 2009 and is currently in operations. According to officials, the spacecraft has experienced some minor electronic sensitivity issues. Specifically, the telescope has "artifacts" in its field of view because of very low level electrical noise in some detector channels. According to project officials, problems of this nature could have been avoided if the project had developed an engineering model, but this would have increased cost and schedule. Project officials indicated that the artifacts in the data will not keep Kepler from meeting its science requirements. None of Kepler's technologies were identified as critical by the project office. All of Kepler's technologies have flown on other missions and were therefore considered heritage. However, the project office acknowledged that the customization of some of Kepler's instruments, and the reliance on heritage technology, proved to be a challenge to Kepler's development. Project officials told us that Kepler's large photometry array added to the complexity of the project because photometers of Kepler's sensitivity have not flown before and proved more difficult to adapt than anticipated—an adaptation that contributed to cost growth. Specifically, development of the focal plane array of the photometer was a challenge because it is the largest ever flown in space and has stringent requirements. We were unable to determine if Kepler's design was stable at its critical design review since drawing counts at the critical design review in October 2006 were unavailable. In addition, Kepler officials told us that the project had difficulty obtaining quality parts as well as parts that satisfy NASA radiation tolerance standards. The project also had difficulty accessing facilities—such as the facilities that put coating on the mirrors—because of the competition among government programs for the facilities and the consolidation of the industrial base. Between its 2007 baseline and March 2009 launch, Kepler experienced a nearly 25 percent increase in development costs and a 9-month increase in schedule. As required by law, NASA reported to the Congress that Kepler exceeded its development cost baseline by more than 15 percent and its schedule baseline by more than 6 months. The project office attributed this to the prime contractor's inability to execute the project's planned activities within the original proposed cost and schedule. Contractor representatives agreed that they underestimated the complexity and the effort required to modify the existing heritage technologies. According to both the Kepler project manager and the contractor's representatives, a $35 million funding cut in fiscal year 2005 significantly contributed to project delays. This funding instability contributed to an overall 20-month delay in the project's schedule and about $169 million in cost growth. In an effort to keep the project executable with sufficient reserves, the project office shortened its operations by 6 months and accepted additional project risk when it canceled or de-scoped several tests. For example, the flight segment vibration test was reduced to an acoustic test, and the vibration tests of the solar panel were eliminated. Additionally, the prime contractor put new management personnel in place and according to contractor representatives, agreed to convert $7.9 million of its projected incentive fee into project reserves held by JPL with the understanding that this money could be earned back for good performance subject to the availability of reserves at the end of development. The contractor will be eligible to earn award fees related to operations. Project Office Comments: The Kepler project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that NASA is extremely pleased with the quality of science coming out of the Kepler mission. [End of Kepler Assessment] Common Name: LDCM: Landsat Data Continuity Mission (LDCM): Figure: Illustration of LDCM. Source: General Dynamics (CAD drawing). [End of figure] The Landsat Data Continuity Mission (LDCM), a partnership between NASA and the U.S. Geological Survey (USGS), seeks to extend the ability to detect and quantitatively characterize changes on the global land surface at a scale where natural and man-made causes of change can be detected and differentiated. It is the successor mission to Landsat 7. The Landsat data series, begun in 1972, is the longest continuous record of changes in the Earth's surface as seen from space. Landsat data is a unique resource for people who work in agriculture, geology, forestry, regional planning, education, mapping, and global change research. Formulation: Formulation start: 10/03; Preliminary design review: 7/09; Project confirmation: 12/09; Implementation: GAO review: 12/09; Critical design review: 3/10; Launch readiness date 12/12. Project Essentials: NASA Center: Goddard Space Flight Center. Partner: U.S. Geological Survey (USGS). Major Contractors: Ball Aerospace and Technologies Corp., General Dynamics Advanced Information Systems, The Hammers Company. Projected Launch Date: December 2012. Launch Location: Vandenberg AFB, Calif. Launch Vehicle: Atlas V. Mission Duration: 5 years (10 years propellant). Project Performance (then year dollars in millions): Preliminary Estimate of Project Life Cycle Cost*: Latest (Oct. 2009): $730 - $800. *This estimate is preliminary, as the project is in formulation and there is still uncertainty in the value as design options are explored. NASA uses these estimates for planning purposes. Launch Schedule: 12/2012. Project Challenges: * Technology Maturity; * Development Partner Performance. Project Status: The project's estimated launch date slipped from July 2011 to December 2012 after a review board reported that the previous development schedule was unachievable. Since then, NASA has directed the LDCM project to proceed with development of the Thermal Infrared Sensor pending an official decision whether that instrument will be included in the mission. Inclusion could add an estimated $160 million to $200 million to development costs. The project received nearly $52 million under the American Recovery and Reinvestment Act of 2009 that will aid development of a thermal infrared sensor and its integration onto the spacecraft. Landsat Data Continuity Mission (LDCM): Detailed Project Discussion: The LDCM instrument payload currently consists of a single science instrument—the Operational Land Imager (OLI). The project is considering the addition of the Thermal Infrared Sensor (TIRS)—a sensor designed to capture thermal information to be used for air quality modeling and wildfire assessment and to operationally monitor water consumption on a field-by-field basis. Although the LDCM project is still awaiting an official decision to include the TIRS instrument in the project, NASA has directed it to proceed with mission development presuming TIRS will be included. The project has begun development of the instrument as it proceeds in the implementation phase, which is scheduled to begin in December 2009 once the project is confirmed. The TIRS project is estimated to cost an additional $160 million to $200 million, and LDCM received $52 million under the American Recovery and Reinvestment Act of 2009 that will assist in incorporating the TIRS instrument. The LDCM project delayed its estimated launch date from July 2011 to December 2012 after it completed its Initial Mission Confirmation Review in September 2008. During this review, the project reported that the previous development schedule was unachievable and increased risk to the mission. The project reported that TIRS should not impact the performance of the OLI or the planned launch date. While the TIRS instrument is a new development effort, many of the subsystems and components were used in earlier flight projects. The project reported that many of the technologies for TIRS were assessed as mature. In September 2009, the project reported that the focal plane array was assessed as mature and that the project has reduced the TIRS development design schedule from 48 months to 39 months, with instrument delivery planned for December 2011. The LDCM project has not reached a design review where we could assess design stability. As of September 2009, the project has released 83 percent of its design drawings. Project officials reported the United States Geological Survey (USGS)— a partner responsible for ground systems elements—is experiencing funding shortfalls that may impair LDCM's ability to meet ground systems requirements for on-orbit verification of instruments and transition to normal operations. The project reports that USGS has taken steps to reduce its funding shortfall through technical redesign, changes in procurement strategy, and some minor exchanges of responsibilities with NASA. Presuming USGS receives its requested funding increase for fiscal years 2011 through 2013, project officials said they believe there should be no impact to LDCM other than an increase in project costs that will be offset by subsequent cost reductions, as NASA assumes some of the responsibilities from USGS. To save costs in the near term, the project reported that USGS has selected a data-processing architecture with heritage from Landsat 7 that provides a good technical solution and reduces cost and schedule risk. Other cost savings include the decision to keep the mission operations center at Goddard Space Flight Center, and opting to procure the services of a flight operations team through an existing contract with NASA. Project Office Comments: The LDCM project office provided technical comments on a draft of this assessment, which were incorporated as appropriate. In addition, the project office commented that the LDCM project is on track for project confirmation in late 2009, and will head into implementation. [End of LDCM Assessment] Common Name: LRO: Lunar Reconnaissance Orbiter (LRO): Figure: Illustration of LRO. Source: LRO Project Office. [End of figure] The Lunar Reconnaissance Orbiter (LRO) is NASA's first mission in its plan to return to the moon and beyond —its Vision for Space Exploration. LRO's mission is to orbit the moon for one year measuring lunar topography, resources, temperatures, and radiation. These data will be used to select a landing site for manned missions to the moon and to ensure astronaut safety. The LRO has a scientific payload of six main instruments and one technology demonstration instrument. LRO's launch vehicle contained a secondary payload, the Lunar Crater Observation and Sensing Satellite (LCROSS), which impacted the Moon to investigate lunar surface volatiles such as water. Formulation: Formulation start: 5/04; Preliminary design review: 2/06; Project confirmation: 5/06; Implementation: Critical design review: 11/06; Launch: 6/09; GAO review: 12/09. Project Essentials: NASA Center Lead: Goddard Space Flight Center. Partners: Boston University, University of California Los Angeles, Southwest Research Institute, Russian Institute for Space Research, Arizona State University, Naval Air Warfare Center. Prime Contractors: in-house development. Launch Date: June 18, 2009. Launch Location: Cape Canaveral AFS, Fla. Launch Vehicle: Atlas V. Mission Duration: 1 year (then science mission). Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $540.1; Latest (Oct. 2009): $590.4; Change: 9.3%. Formulation Cost: Baseline Est. (FY 2009): $93.3; Latest (Oct. 2009): $94.4; Change: 1.2%. Development Cost: Baseline Est. (FY 2009): $420.8; Latest (Oct. 2009): $473.1; Change: 12.4%. Operations Cost: Baseline Est. (FY 2009): $25.8; Latest (Oct. 2009): $22.9; Change: -11.2%. Launch Schedule: Baseline Est. (FY 2009): 10/2008; Latest (Oct. 2009): 6/2009; Change: 8 months. Project Challenges: * Complexity of Heritage Technology; * Launch Manifest. Project Status: LRO successfully launched on June 18, 2009, after an 8-month delay, and is currently operating in lunar orbit. Lunar Reconnaissance Orbiter (LRO): Detailed Project Discussion: The project did not identify any critical technologies. Each of the project's major instruments is based significantly on heritage technology. However, the project manager said the project had underestimated the difficulty of the modifications needed. For example, the project manager said the Lunar Reconnaissance Orbiter cameras needed some technical work to adapt designs for the lunar thermal environment as well as some redesign when areas needing reinforcement were found during testing. The lunar orbiter laser altimeter, while similar to laser altimeters that have flown on previous Mars and Mercury missions, had issues with the electronics that time the laser pulses of the altimeter, which, according to the project manager, took more time to resolve than originally expected. The Diviner Lunar Radiometer Experiment instrument is almost a copy of an instrument on Mars now, but it experienced motor failures in testing, which the project manager said took extra time and money to recover from. Finally, the Lyman-Alpha Mapping Project instrument, a copy of the Pluto Alice instrument, was slightly delayed because of a detector failure during thermal vacuum testing. According to the project manager, most instruments required additional design and analysis of their thermal control designs to operate reliably on the mission. Redesign was necessary because the lunar environment presents a harsher thermal environment than the environment faced by earth- orbiting missions. The project did not measure design stability by percentage of drawings completed at the critical design review (CDR), and therefore, we did not assess design stability. LRO successfully launched on June 18, 2009, and, according to the project manager, entered lunar orbit with roughly three times the amount of fuel the program had planned on having at that point, which may allow for an extended mission. LRO's launch, however, was delayed 8 months from October 2008 because of several factors. Initially, the project delayed launch for one month to accommodate problems with the ground data system software, reduced schedule slack, and improved launch window opportunities. NASA then accepted a request from United Launch Alliance to swap launch positions with a non-NASA mission, moving the launch date to March 2009. Delays in the launch manifest since then caused LRO's launch date to slip into June 2009. The project manager reported that the project team used the 8-month schedule delay to further mitigate technical risks. As a result, the project fully investigated identified issues and made hardware changes. For example, the LRO Wide Angle Camera was replaced during this time. As a result of the launch delays, the project's development cost increased by over $52 million, or 12.4 percent. Project Office Comments: The LRO project office provided technical comments on a draft of this assessment, which were incorporated as appropriate. [End of LRO Assessment] Common Name: MMS: Magnetospheric Multiscale (MMS): Figure: Illustration of MMS. Source: MMS Project Office (Computer Model). [End of figure] The Magnetospheric Multiscale (MMS) is made up of four identically instrumented spacecraft. The mission will use the Earth's magnetosphere as a laboratory to study the microphysics of magnetic reconnection, energetic particle acceleration, and turbulence. Magnetic reconnection is the primary process by which energy is transferred from solar wind to Earth's magnetosphere and is the critical physical process determining the size of a space weather storm. The four spacecraft will be launched together in a stacked configuration, and then fly in a tetrahedral (pyramid) formation, adjustable over a range of 10 to 400 kilometers, enabling them to capture the three-dimensional structure of the reconnection sites they encounter. Formulation: Formulation start: 5/02; Preliminary design review: 5/09; Project confirmation: 6/09; Implementation: GAO review: 12/09; Critical design review: 8/10; Launch readiness date 10/14. Project Essentials: NASA Center Lead: Goddard Space Flight Center. International Partners: Austria, France, Japan, Sweden. Major Contractors: Southwest Research Institute. Projected Launch Date: October 2014. Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Atlas V. Mission Duration: 2 years. Project Performance (then year dollars in millions) Total Project Cost: Latest* (Oct. 2009): Not provided. Formulation Cost: Latest* (Oct. 2009): Not provided. Development Cost: Latest* (Oct. 2009): Not provided. Operations Cost: Latest* (Oct. 2009): Not provided. *NASA suggested it will supply the baseline estimates for this project when it provides them to Congress in the FY11 budget. Launch Schedule: 10/2014. Project Challenges: * None Currently Identified. Project Status: MMS was approved for implementation in July 2009 after being in formulation for 7 years, due in part to budget cuts to the Solar Terrestrial Probes program and in part from the difficulty of developing a new spacecraft. NASA has not yet provided a cost baseline. Initial cost estimates for the project in 2002 were $369 million, but the new life cycle cost baseline will likely exceed $900 million because of the need for a larger instrument suite and multiple spacecraft. The project is currently scheduled to launch in October 2014. Magnetospheric Multiscale (MMS): Detailed Project Discussion: MMS is a classic research mission and was ranked as the highest priority moderate-sized mission in the 2003 Solar and Space Physics Decadal survey of the National Research Council. Due in part to the groundbreaking nature of this mission, NASA and the project's major contractor have partnered with several other countries, including Austria, France, Japan, and Sweden. These countries are contributing several instruments to the project as well as engineering support and test facilities. There is no exchange of money between NASA and the foreign governments, as each will pay for its respective contribution. At the preliminary design review, the MMS project assessed both of its critical technologies and three of its five heritage technologies as mature. The two remaining heritage technologies—the four pound thrusters for large maneuvers and the payload separation system—will start testing and modifications after their contracts are awarded in 2009. According to the project manager, the four pound thrusters should be available "off the shelf," but will require testing to ensure they are compatible with a spinning spacecraft such as MMS. The project also reported that flight-proven thrusters will need to be qualified to the MMS firing cycle, which is different than that of the heritage missions. In addition, the project manager told us that the payload separation system was not procured from the launch vehicle provider because of the high cost. Instead, the project has decided to build its own separation system based on heritage flight-proven systems which will need to be modified to meet the grounding requirements for MMS. The MMS project has not reached a design review where we could assess design stability. The project did not formally release design drawings at the preliminary design review. Project officials told us that they are preparing for the critical design review (CDR) by completing flight and ground system design and analysis, completing development and test of spacecraft and instrument engineering models, conducting peer reviews, and conducting instrument- and spacecraft-level CDR's. A project official added that having 70 to 80 percent of design drawings completed by CDR is normal, but officials have not established any goals for the project. Despite being authorized to enter implementation in June 2009, NASA has not yet provided a cost baseline for MMS. However, the project manager indicated that the life-cycle cost would be at least $900 million. The project was authorized to enter formulation in 2002 with an initial cost estimate of $369 million. The project manager said initial cost estimates were for a smaller instrument suite than what is currently planned for the mission and added that cost drivers for the project since the initial cost estimates included the requirement for magnetic and electrostatic cleanliness and the need for multiple spacecrafts. Additionally, MMS was in formulation for about 7 years due in part to budget cuts to the Solar Terrestrial Probes program. Additionally, in 2005 it was determined that the original approach to use an off-the-shelf spacecraft bus was not viable, and in 2006, NASA assigned the development for the MMS spacecraft to Goddard Space Flight Center. Project Office Comments: The MMS project office provided technical comments on a draft of this assessment, which were incorporated as appropriate. The project office also commented that during a June 2009 review with the NASA Administrator, the project was approved to enter implementation and establish a cost and schedule baseline. In addition, project officials believe MMS is currently on schedule in its development of the detailed mission design, with no significant unresolved challenges heading toward the CDR in August 2010. [End of MMS Assessment] Common Name: MSL: Mars Science Laboratory (MSL): Figure: Illustration of MSL. Source: NASNJPL-Caltech. [End of figure] The Mars Science Laboratory (MSL) is part of the Mars Exploration Program (MEP). The MEP seeks to understand whether Mars was, is, or can be a habitable world. To answer this question the MSL project will investigate how geologic, climatic, and other processes have worked to shape Mars and its environment over time, as well as how they interact today. The MSL will achieve these objectives by placing a mobile science laboratory on the Mars surface to assess a local site as a potential habitat for life, past or present. The MSL is considered one of NASA's flagship projects and will be the most advanced rover yet sent to explore the surface of Mars. Formulation: Formulation start: 11/03; Preliminary design review: 6/06; Project confirmation: 8/06; Implementation: Critical design review: 6/07; GAO review: 12/09; Launch readiness date 10/11. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. Partners: U.S. Department of Energy, Centre Nationale d'Etude Spatiale (France), Russian Federal Space Agency, Centro de Astrobiologia (Spain), Canadian Space Agency. Major Contractors: in-house development. Projected Launch Date: October 2011. Launch Location: Cape Canaveral AFS, Fla. Launch Vehicle: Atlas V. Mission Duration: 1 year of travel, 2 years of operations. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2008): $1,642.2; Latest (Oct. 2009): $2,305.3; Change: 40.4%. Formulation Cost: Baseline Est. (FY 2008): $515.1; Latest (Oct. 2009): $515.5; Change: 0.1%. Development Cost: Baseline Est. (FY 2008): $968.6; Latest (Oct. 2009): $1,631.0; Change: 68.4%. Operations Cost: Baseline Est. (FY 2008): $158.5; Latest (Oct. 2009): $158.8; Change: 0.2%. Launch Schedule: Baseline Est. (FY 2008): 9/2009; Latest (Oct. 2009): 10/2011; Change: 25 months. Project Challenges: * Technology Maturity; * Complexity of Heritage Technology; * Design Stability. Project Status: Since the project was baselined, MSL's cost has grown over $660 million because of technological and engineering problems. This includes more than a 68 percent increase in development costs. The project is using a 25-month schedule delay to work on overcoming technical challenges with the actuators and avionics that were the primary drivers for the slip. NASA reported to the Congress that MSL had exceeded both its development cost and schedule baselines. In addition, MSL is currently seeking re-authorization from the Congress since the project has exceeded its cost baseline by more than 30 percent. Mars Science Laboratory (MSL): Detailed Project Discussion: At the preliminary design review, the project assessed all seven of its critical technologies as immature resulting from late development challenges it encountered. At the critical design review a year later, three of the seven critical technologies had been replaced by backup technologies with two of the seven still assessed as immature, including one of the replacement technologies. In addition, the MSL project relied on several heritage technologies that had to be re- designed, re-engineered, or replaced. For example, the heat shield— constructed of a super light-weight ablator that had flown on previous missions—was considered nearly ready at the critical design review but experienced a significant setback in testing and could not be approved for use on MSL. As a result, the project selected a new and less mature technology—phenolic impregnated carbon ablator (PICA)—which was successfully used on the STARDUST mission Earth return capsule. According to the MSL project office, the impact of this change was approximately $30 million in cost growth and a nine-month delay in delivery of the heat shield. The MSL project design was not stable at the critical design review (CDR). Several design changes were required to address various issues. For example, the plumbing for the propulsion system was redesigned because it was determined that MSL needed larger, rigid lines for the system than were previously used on smaller Mars rovers. These thicker lines inadvertently became load-bearing components, which caused the project to redesign part of the structure to account for the loads and shift them to MSL's primary structure. Furthermore, project officials said they underestimated the overall complexity of the rover and realized in 2008 that MSL could not maintain a 2009 launch readiness date. The project experienced problems with the actuators—the motors that allow the vehicle to move and execute the sample operations performed by the lab. MSL project officials said they wanted to implement a dry lubrication scheme with lightweight titanium gears for the actuators. However, during fabrication it was discovered that this scheme did not provide the durability needed for MSL. The project reverted to a heavier stainless steel gear system with the same wet lubricant used by prior projects. Project officials added that this decision to change the actuator scheme was late in the process, ultimately causing delays when one of the vendors developing the stainless steel gears could not meet production demands. In addition, project officials stated that the avionics package was also part of the reason for the launch delay. They said that the avionics hardware was a new design that had never been flown on earlier missions and was delivered to the project in an immature state. The delay in development of the avionics hardware resulted in delays to the related avionics software. Project officials told us they hope to have these issues resolved by November 2009 and that they plan to perform all the necessary test and integration activities for the spacecraft in 2010. They added that extra time will allow for a much more robust test campaign. Since the baseline in 2008, the life-cycle cost for the project has increased by over $660 million—including more than a 68 percent increase in development costs—and the launch has been delayed until at least October 2011 since launch windows for Mars mission are optimally aligned every 26 months. As a result, NASA reported to the Congress, as required by law, that MSL had exceeded its development cost baseline by more than 15% and schedule baseline by more than 6 months. In addition, NASA is seeking re-authorization from Congress since the project has exceeded its development cost baseline by more than the 30 percent. Project Office Comments: The MSL project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. [End of MSL Assessment] Common Name: NPP: NPOESS Preparatory Project (NPP): Figure: Illustration of NPP. Source: Ball Aerospace. [End of figure] The NPOESS Preparatory Project (NPP) is a joint mission with the National Oceanic and Atmospheric Administration and the U.S. Air Force. The satellite will measure ozone, atmospheric and sea surface temperatures, land and ocean biological productivity, and cloud and aerosol properties. The NPP mission has two objectives. First, NPP will provide a continuation of global weather observations following the Earth Observing System missions Terra and Aqua. Second, NPP will provide the National Polar-orbiting Operational Environmental Satellite System (NPOESS) with risk-reduction demonstration and validation for the critical NPOESS sensors, algorithms, and ground data processing. Formulation: Formulation start: 11/98; Preliminary design review: 1/03; Critical design review: 8/03. Implementation: Project confirmation: 11/03; GAO review: 12/09; Launch readiness date 1/11. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $672.8; Latest (Oct. 2009): $799.4; Change: 18.8%. Formulation Cost: Baseline Est. (FY 2009): $47.3; Latest (Oct. 2009): $47.7; Change: 0.8%. Development Cost: Baseline Est. (FY 2009): $593.0; Latest (Oct. 2009): $725.1; Change: 22.3%. Operations Cost: Baseline Est. (FY 2009): $32.5; Latest (Oct. 2009): $26.6; Change: -18.2%. Launch Schedule: Baseline Est. (FY 2009): 4/2008; Latest (Oct. 2009): 1/2011; Change: 33 months. Project Essentials: NASA Center Lead: Goddard Space Flight Center. Partner: National Atmospheric and Oceanic Administration and U.S. Air Force. Major Contractors: Northrop Grumman Electrical Systems and Ball Aerospace and Technologies Corp. Projected Launch Date: January 15, 2011. Launch Location: Vandenberg AFB, Calif. Launch Vehicle: Delta II. Mission Duration: 5 years. Project Challenges: * Technology Maturity; * Complexity of Heritage Technology; * Design Stability; * Development Partner Performance. Project Status: Due primarily to the late delivery of two key instruments being developed by the project partners, the NPP has experienced over $132 million in development cost growth and a 33-month delay in its launch readiness date. As a result, NASA has reported to the Congress that the NPP project has exceeded both its development cost and schedule baselines. The project is currently slated to launch in January 2011— although continuing problems and delays put this launch date at risk. NPOESS Preparatory Project (NPP): Detailed Project Discussion: The NPP project office identified six critical technologies for the project—the spacecraft and all five instruments. Five of the six critical technologies were assessed as immature at the preliminary and critical design reviews. The NPP project now considers all critical technologies to be mature. However, the project reports an inability to reduce risks to an acceptable level on three instruments, including the Visible Infrared Imaging Radiometer Suite (VIIRS), the Cross-track Infrared Sounder (CrIS), and the Ozone Mapper Profiler Suite. NASA officials told us they lack confidence in the processes used by the NPOESS Integrated Program Office (IPO), which is composed of National Oceanic and Atmospheric Administration and Department of Defense officials, in developing the instruments and are unsure how the instruments will function on orbit. Therefore, NPP will be launched with significant residual risk against mission success, including the potential for a gap in continuity or degraded capability. Furthermore, project officials report that these instruments have failed tests and had difficulties meeting mission science requirements. Management and developmental partner challenges have resulted in cost overruns and schedule delays in the VIIRS and CrIS instruments. VIIRS began thermal vacuum testing in early May 2009; however, continued slow test execution and problems during environmental testing have led to further delays in its delivery to the NPP integration contractor. The instrument is now scheduled to be delivered in December 2009. Additionally, testing of the CrIS instrument found problems such as a faulty calibration target and overstressed semiconductors, which led to delays in its production. According to the project manager, the CrIS instrument was supposed to be delivered in 2008 but is now slated to be delivered by late spring 2010. The project is currently slated to launch in January 2011—although continuing problems and delays threaten this launch date. Since this will be one of the last missions to be launched on a Delta II, availability of trained personnel to launch NPP may be limited. The NPP project design was not stable at the critical design review (CDR). Both the CrIS and the VIIRS had to be redesigned because of failures that were detected during testing after the CDR. Project officials said a 31 percent increase in new engineering drawings was largely attributed to the redesign of the VIIRS and CrIS stemming from testing failures. Since NPP was baselined in fiscal year 2007, the project's development cost has increased 22 percent, and its schedule has increased by 33 months. As a result, NASA has reported to the Congress, as required by law, that the NPP project has exceeded its development cost baseline by more than 15 percent and its schedule baseline by more than 6 months. The project office attributes almost all of the cost and schedule changes to the late delivery of the VIIRS instrument by the project partners. To manage NPP cost increases, the NPOESS program halted or delayed activities on other components—including a sensor planned for another satellite—and redirected those funds to the VIIRS and CrIS instruments. Furthermore, because NPOESS is now not scheduled to launch until 2014, NPP will not be the research satellite it was originally intended to be; instead, it will have to function as an operational satellite until the first NPOESS satellite is launched. Project Office Comments: The NPP project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. In addition, project officials commented that they agree with GAO that delays continue to be experienced because of issues with development of instruments by the project's partners. [End of NPP Assessment] Common Name: CEV: Orion Crew Exploration Vehicle (CEV): Figure: Illustration of CEV. Source: Lockheed Martin Space Systems (artist depiction). [End of figure] NASA's Orion Crew Exploration Vehicle (CEV), as part of the Constellation Program, is designed to be the next-generation of spacecraft to carry crew and cargo to the International Space Station (ISS) and to the Moon. The initial Constellation Program includes the Orion and the Ares I system that are expected to replace the Space Shuttle, which is slated to retire in 2010. The Orion CEV is to be launched by the Ares I Crew Launch Vehicle. Plans call for Orion to carry four astronauts to the International Space Station (ISS) and to the Moon after linking up with an earth departure stage. The capsule will return to Earth and descend on parachutes to the surface. Orion has three main elements—the crew module (capsule), service module/spacecraft adapter, and launch abort system. Formulation: Formulation start: 7/06; Preliminary design review: 8/09; GAO review: 12/09. Implementation: Critical design review: 2/11; Initial operational capability: 3/15. Project Essentials: NASA Center Lead: Johnson Space Center. International Partner: None. Major Contractors: Lockheed Martin. Projected Launch Date: March 2015. Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Ares I. Mission Duration: Varied based on destination. Project Performance (then year dollars in millions): Preliminary Estimate of Project Life Cycle Cost*: Latest (Oct. 2009): $20,000 to $29,000. * This estimate is preliminary, as the project is in formulation and there is still uncertainty in the value as design options are explored. NASA uses these estimates for planning purposes. This estimate is for the Orion vehicle only. Launch Schedule: 3/2015. Project Challenges: * Complexity of Heritage Technology; * Funding Issues. Project Status: Technical challenges, mass growth, and design issues delayed Orion's preliminary design review by 12 months to August 2009. In December 2008, NASA decided to defer work on Orion's lunar mission requirements in order to focus on developing a vehicle that can fly to the ISS. The Orion project received nearly $166 million under the American Recovery and Reinvestment Act of 2009 that will be used for technology development and reduction of technical risks. NASA delayed the first crewed flight from September 2014 to March 2015 to increase schedule confidence in program cost and schedule goals. Orion Crew Exploration Vehicle (CEV): Detailed Project Discussion: The Orion project identified one critical technology for the spacecraft: the thermal protection system, or heatshield. This is a heritage technology from the Apollo program and is required for the spacecraft to survive reentry from earth orbit. However, it may be difficult to repeatedly manufacture to consistent standards because the heatshield uses a framework with many honeycomb shaped cells, each of which must be individually filled with no voids or imperfections. In addition, development of the launch abort system, which would pull the Orion capsule away from the Ares I launch vehicle in the case of a catastrophic problem during launch, remains a high risk area even though it was not identified as a critical technology. A year after the initial contract was awarded, the first launch abort system subcontractor did not have a viable design and was replaced. Furthermore, continued weight growth and requirements changes are contributing to instability in the Orion design For example, in its efforts to reduce the mass of the Orion vehicle, NASA chose to move from land nominal landing to water nominal landing to reduce mass by eliminating air bags and, according to officials, by reducing the number of parachutes. To increase its level of confidence in the Constellation program baseline, NASA has delayed the first crewed flight from September 2014 to March 2015 and deferred work on a vehicle that can meet the lunar mission requirements so that NASA can focus its efforts on developing a vehicle that can fly the ISS mission. NASA's original strategy for the Orion project was to develop one vehicle capable of supporting both ISS and lunar missions. This new phased approach, however, could require two qualification programs for the Orion vehicle—one pre-2015 Orion qualification program for ISS mission requirements and a second post-2015 Orion qualification program for lunar mission requirements. According to the Constellation program manager, the Constellation program's poorly phased funding plan has diminished the Orion project's ability to deal with technical challenges. NASA initiated the Constellation program relying on the accumulation of a large budget reserve in fiscal years 2006 and 2007 to fund activities in fiscal years 2008 through 2010. Thereafter, NASA anticipated that the retirement of the space shuttle program in 2010 would free funding for the Constellation program. The program's risk management system identified this strategy as high risk, warning that shortfalls could occur in fiscal years 2009 through 2012. NASA has not released official cost and schedule estimates to complete the Orion project. NASA officials stated that these estimates will be made available when the project moves into implementation, or at the conclusion of the Constellation Program's non-advocate review. However, the value of the development contracts for Orion has increased by $2.5 billion since 2006. The Constellation program received $400 million under the American Recovery and Reinvestment Act of 2009, of which the Orion project is slated to receive nearly $166 million that will help the project mitigate technical challenges. Nevertheless, in September 2009, an independent commission formed by the President to study the future of U.S human spaceflight reported that NASA's plans for the Constellation program to return man to the moon by 2020 are unexecutable without drastic increases to NASA's current budget profile. Project Office Comments: The Orion project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that they believe NASA will continue to narrow the trade-space options as the project moves through the formulation phase and toward a confirmed baseline design. They also believe that steady progress has been made in all technology areas and that the project has achieved stability in requirements. Project officials commented that Orion met the mass requirements for the ISS phase at their preliminary design review, and that the mass trend is favorable toward meeting lunar phase requirements. [End of CEV Assessment] Common Name: RBSP: Radiation Belt Storm Probes (RBSP): Figure: Illustration of RBSP. Source: NASA/Johns Hopkins University Applied Physics Laboratory. [End of figure] The RBSP mission will help us understand the Sun's influence on the Earth and near-Earth space by studying the planet's radiation belts at various scales of space and time. This insight into the physical dynamics of the Earth's radiation belts will provide scientists data to make predictions of changes in this critical region of space. Understanding the radiation belt environment has practical applications in the areas of spacecraft system design, mission planning, spacecraft operations, and astronaut safety. The two spacecraft will measure the particles, magnetic and electric fields, and waves that fill geospace and provide new knowledge on the dynamics and extremes of the radiation belts. Formulation: Formulation start: 9/06; Preliminary design review: 10/08; Project confirmation: 12/08; Implementation: GAO review: 12/09; Critical design review: 12/09; Launch readiness date 5/12. Project Essentials: NASA Center Lead: Goddard Space Flight Center. Partner: National Reconnaissance Office. Major Contractors: John Hopkins University - Applied Physics Laboratory, Boston University. Projected Launch Date: May 18, 2012. Launch Location: Cape Canaveral, Fla. Launch Vehicle: Atlas V. Mission Duration: 2 years. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $685.8; Latest (Oct. 2009): $685.8; Change: 0.0%. Formulation Cost: Baseline Est. (FY 2009): $88.2; Latest (Oct. 2009): $88.2; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $533.9; Latest (Oct. 2009): $533.9; Change: 0.0%. Operations Cost: Baseline Est. (FY 2009): $63.7; Latest (Oct. 2009): $63.7; Change: 0.0%. Launch Schedule: Baseline Est. (FY 2009): 5/2012; Latest (Oct. 2009): 5/2012; Change: none. Project Challenges: * None Currently Identified. Project Status: In January 2009, the RBSP project was confirmed and established its cost and schedule baselines. At that time, NASA added $54 million and seven months to the project's cost and schedule to ensure the project entered implementation with a higher confidence of mission success. Radiation Belt Storm Probes (RBSP): Detailed Project Discussion: Project officials reported that there are no new critical technologies for RBSP. They did identify two heritage technologies—the chip-to- board bonding techniques on the integrated circuits for the spacecraft transceiver and the method used to measure secondary particles in the Helium-Oxygen-Proton-Electron (HOPE) instrument—both of which the project assessed as mature at the mission preliminary design review in October 2008. Project officials indicated that one of the primary challenges for RBSP is developing a spacecraft capable of withstanding high levels of radiation that it will encounter during the mission. They added that RBSP is incorporating other heritage technologies developed for the Juno mission—which has an even tougher radiation environment— and they are confident RBSP will be able to withstand high radiation levels. The project reported potential delays in procuring detectors from a project contractor, which could delay the delivery of two instruments— the Magnetic Electron Ion Spectrometer, and the Relativistic Electron Proton Telescope. NASA is monitoring the progress of the flight detectors to ensure timely delivery. In addition, project officials identified a project risk concerning the four wire booms that protrude from each spacecraft since the project will not be able to test them in the deployed configuration; however, they also indicated that they have extensive experience with similar booms. Additionally, the project reported that NASA recently provided instructions that prohibited the use of certain connectors as part of their ongoing monitoring of quality parts and qualification standards, which caused the project to review the type of connectors used in the observatory and replace the connectors as applicable. Although the project classifies the likelihood of an in-flight failure due to the prohibited connectors as very small, possible consequences include loss of the spacecraft or an instrument. The RBSP project has not reached a design review where we could assess design stability. As of September 2009 project officials plan on releasing 87 percent of design drawings by the critical design review in December 2009. In mid-2009, the project reported that some spacecraft subsystems were behind schedule because of delays in drawing releases and lack of available resources. The project took several steps to get back on schedule, including the addition of personnel for subsystem schedule management. Subsequently, the project has released engineering model drawings for three of the four subsystems that were behind schedule, and plans on releasing drawings for the fourth subsystem in September 2009. The project established a baseline of $685.8 million and a May 2012 launch date in January 2009. This included the addition of seven months to the project's schedule and $54 million to the life-cycle cost at project confirmation to ensure the project entered implementation with a higher confidence of mission success. Project Office Comments: The RBSP project office provided technical comments on a draft of this assessment, which were incorporated as appropriate. In addition, the RBSP project office commented that they have made significant progress in development of engineering models for spacecraft subsystems and science instruments, delivery of key parts, and release of flight model drawings, which is expected to continue through the mission critical design review. [End of RBSP Assessment] Common Name: SDO: Solar Dynamics Observatory (SDO): Figure: Illustration of SDO. Source: SGT Inc. [End of figure] NASA's Solar Dynamics Observatory (SDO) will investigate how the Sun's magnetic field is structured and how its energy is converted and released into the heliosphere in the forms of solar wind, energetic particles, and variations in solar irradiance. The primary goal of the SDO mission is to understand the solar variations that influence life on Earth and humanity's technological systems. It seeks to do this by determining how the Sun's magnetic field is generated and structured, and how this stored magnetic energy is released. Analysis of data from SDO's three instruments—Atmospheric Imaging Assembly (AIA), Extreme Ultraviolet Variability Experiment (EVE), and Helioseismic and Magnetic Imager (HMI)—will improve the science needed to enable space weather predictions. Formulation: Formulation start: 12/01; Preliminary design review: 3/04; Project confirmation: 6/04; Critical design review: 4/05; Implementation: GAO review: 12/09; Launch readiness date 2/10. Project Essentials: NASA Center Lead: Goddard Space Flight Center. International Partner: None. Major Contractors: Stanford University, Lockheed Martin Solar Astrophysics Laboratory, University of Colorado. Projected Launch Date: February 3, 2010. Launch Location: Kennedy Space Center, Fla. Launch Vehicle: Atlas V. Mission Duration: 5 years (10 year goal). Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $787.5; Latest (Oct. 2009): $867.6; Change: 10.5%. Formulation Cost: Baseline Est. (FY 2009): $78.0; Latest (Oct. 2009): $84.9; Change: 8.8%. Development Cost: Baseline Est. (FY 2009): $623.7; Latest (Oct. 2009): $682.6; Change: 9.4%. Operations Cost: Baseline Est. (FY 2009): $83.8; Latest (Oct. 2009): $100.1; Change: 19.5%. Launch Schedule: Baseline Est. (FY 2009): 8/2008; Latest (Oct. 2009): 2/2010; Change: 18 months. Project Challenges: * Complexity of Heritage Technology; * Design Stability; * Contractor Performance; * Development Partner Performance; * Funding Issues; * Launch Manifest. Project Status: The project's inability to meet the August 2008 launch date has resulted in cost increases and additional schedule delays prompting NASA to report to the Congress that the SDO project exceeded its development schedule baseline. Due to a crowded launch manifest, the next available launch date for SDO is February 2010, resulting in an 18-month schedule delay. Solar Dynamics Observatory (SDO): Detailed Project Discussion: The SDO project reported that its only critical technology—a 4K x 4K array of charge-coupled devices (CCD) to be used in both the HMI and AIA instruments—was mature at the project's preliminary design review. The United Kingdom originally led development of the CCD camera systems, but dropped out of the project before the preliminary design review. Project officials stated that SDO was purposefully designed to use existing technology components, but said they also recognized that some technologies—such as the Ka-band transmitter, high-speed bus, and high-gain antenna system—required modifications to be used on SDO. For example, the existing technology for the Ka-band transmitter, which the project assessed as immature at the preliminary design review, required a new design for integration with SDO. Project officials told us that Northrop Grumman originally was to build the Ka-band transmitter, but its development was brought in house after contractor performance issues arose. SDO's design was not stable at the critical design review (CDR). Following this review, the project experienced nearly a 1,200 percent overall increase in the number of releasable drawings expected. Project officials said only drawings for in-house structures, such as propulsion systems, electronics, instrument ports, the high-gain antenna system, and the spacecraft, were considered at CDR. Drawings for the instruments were not included and flight drawings were only in draft form at CDR. The project estimated it released less than 63 percent of engineering drawings at each of their instrument-level CDRs. Project officials indicated that flight drawings did not need to be ready so far in advance of the project's launch readiness date since there was enough time to build these components. SDO also experienced several problems during testing of flight hardware. The project suffered a technical setback in 2007 when the thermal vacuum chamber being used to test the high gain antenna overheated, resulting in the need to completely rebuild the antenna. Several risks to the project were identified during testing. For example, testing identified a part on the spacecraft's high-speed bus that, under certain circumstances, could cause the spacecraft to reset itself, which could mean failure to meet science data quality and completeness requirements. Project officials indicated that a software update to the high-speed bus should correct this issue. At the time of CDR in April 2005, the SDO project budget was reduced by one-third for fiscal year 2005 because of other funding priorities. As a result, the project underwent a re-plan that delayed the project's launch readiness date from April 2008 to August 2008. Subsequent scheduling issues for testing of the AIA instrument and other spacecraft parts problems caused further delays and cost increases and the launch date slipped to December 2008, resulting in a cost increase of $18.1 million. Because of launch manifest issues, SDO's launch date has since slipped to February 2010 and approximately $50 million was added to the project's life-cycle costs from the previous year, which is largely attributable to keeping project staff longer than expected and conducting additional spacecraft tests. As required by law, NASA has reported to the Congress that the SDO project has exceeded its development schedule baseline by more than 6 months. Project Office Comments: The SDO project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. Project officials also commented that the crowded launch manifest has been the single reason for the delays that have occurred since December 2008 and since that time the project has waited for a firm launch slot. The officials added that the observatory was shipped to Florida in July 2009 and is on track for a February 2010 launch opportunity. In addition, NASA officials state that an independent review team determined that SDO's critical design review was successful and that the technical baseline was solid. [End of SDO Assessment] Common Name: SOFIA: Stratospheric Observatory for Infrared Astronomy (SOFIA): Figure: Illustration of SPOHIA: Source: SOFIA Program Office. [End of figure] SOFIA is a joint project between NASA and the German Space Agency (DLR) to install a 2.5 meter telescope in a specially modified Boeing 747SP aircraft. This airborne observatory is designed to provide routine access to the visual, infrared, far-infrared, and sub- millimeter parts of the spectrum. Its mission objectives include studying many different kinds of astronomical objects and phenomena, including star birth and death; the formation of new solar systems; planets, comets, and asteroids in our solar system; and black holes at the center of galaxies. Interchangeable instruments for the observatory are being developed to allow a range of scientific measurement to be taken by SOFIA. Formulation: Formulation start: 10/91. Implementation: GAO review: 12/09; Initial operational capability: 3/10; Full operational capability: 12/14. Project Essentials: NASA Center Lead: Dryden Flight Research Center. International Partner: German Space Agency (DLR). Major Contractors: L3 Communications, MPC Products Corporation, University Space Research Association. Projected Operational Capability: March 2010. Aircraft: Modified 747SP. Sortie Location: Dryden Flight Research Center, Calif. Mission Duration: 20 years of science mission flights. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $2,954.5; Latest (Oct. 2009): $2,960.2; Change: 0.0%. Formulation Cost: Baseline Est. (FY 2009): $35.0; Latest (Oct. 2009): $35.0; Change: 0.0%. Development Cost: Baseline Est. (FY 2009): $919.5; Latest (Oct. 2009): $1,081.8; Change: 17.7%. Operations Cost: Baseline Est. (FY 2009): $2,000.0; Latest (Oct. 2009): $1,843.4; Change: -7.8%. Launch Schedule: Baseline Est. (FY 2009): 12/2013; Latest (Oct. 2009): 12/2014; Change: 12 months. Project Challenges: * Complexity of Heritage Technology; * Contractor Performance; * Funding Issues. Project Status: Initial science flights have slipped from August 2009 to no earlier than March 2010. SOFIA's current development costs are estimated to be about $1.08 billion, representing a more than 300 percent increase from the original estimate of $251 million in 1997. This includes a $400 million cost increase from fiscal year 2009, which project officials said is due primarily to NASA choosing not to secure additional international partners for the project. NASA has reported to the Congress that SOFIA exceeded both its cost and schedule baselines. Stratospheric Observatory for Infrared Astronomy (SOFIA): Detailed Project Discussion: We could not assess the technology maturity or the design stability of the overall project as NASA did not provide information related to the aircraft modification. Data provided for development of the instruments that will fly on SOFIA generally indicates a high level of technology maturity. Many of these technologies have already been used on ground-based telescopes, and the early instruments are essentially finished and are waiting for the observatory to be completed. Of the eight first-generation science instruments, three are ready for installation, one will be ready for installation upon completion of airworthiness documentation, and the others are scheduled to be ready between 2010 and 2012. Similarly, we were unable to determine design stability of the instruments since the drawings were still preliminary at the critical design review. Design work on SOFIA is 97 percent complete, but several subsystems will still be in design into 2011. The SOFIA project experienced problems related to the original prime contractor's performance early in development. At this time, the contract required the contractor to perform significant project management activities. According to project officials, that contractor had neither the project management experience nor the design-build expertise necessary for the project. Consequently, NASA reduced the contractor's management role for both development and operations of SOFIA and subsequently utilized government personnel to perform these functions in-house. The contractor also experienced challenges with modification of the aircraft used for SOFIA, which led to significant cost overruns. Project officials said this contractor, who was also responsible for the aircraft's modification and integration, had limited experience with this type of work and did not fully understand the statement of work, which resulted in cost overruns. Since the December 2007 flight test, testing of SOFIA has stayed mostly on schedule. However, the first open-door flight test was delayed 8 months and successfully took place in December 2009. This delay was due mostly to the development of the Cavity Door Drive System (CDDS) controller which was lagging due to poor performance by the contractor and integration issues. The CDDS contractor experienced problems stemming from poor workmanship, which project officials claim is partially attributable to a shrinking industrial base. In addition, testing for the High-speed Imaging Photometer for Occultation (REPO) revealed that some activities the project thought could be done in parallel were more difficult than expected and must instead be done serially. Subsequently, initial science flights have slipped from August 2009 to no earlier than March 2010, and given the various challenges, project officials said that science flights may be delayed even further. As a result of ongoing cost growth early in development, the SOFIA project was slated for cancellation in 2006. However, later that year the project was reinstated. It was re-baselined in July 2007. The fiscal year 2010 budget request showed the project's life-cycle cost increased by $400 million from fiscal year 2009, which project officials said is due primarily to NASA choosing not to secure additional international partners for cost-sharing on the project. SOFIA's current development costs are estimated to be about $1.08 billion, representing a more than 300 percent increase from the original estimate of $251 million in 1997. As required by law, NASA has reported to the Congress that SOFIA exceeded its development cost baseline by more than 15 percent and its schedule baseline by more than 6 months. Project Office Comments: The SOFIA project provided technical comments to a draft of this assessment, which were incorporated as appropriate. The project office also commented that the SOFIA project continues to make good progress toward science flight operations. [End of SOPHIA Assessment] Common Name: WISE: Wide-field Infrared Survey Explorer (WISE): Figure: Illustration of WISE. Source: NASNJPL-Caltech (artist depiction). [End of figure] The WISE mission is designed to map the sky in infrared light and search for the nearest and coolest stars, the origins of stellar and planetary systems, the most luminous galaxies in the universe, and most main-belt asteroids larger than 3 kilometers. It is also intended to create a catalog of over 300 million sources that will be of interest to future infrared studies, including the upcoming James Webb Space Telescope mission. During its 6-month mission, WISE will use a four-channel imager to take overlapping snapshots of the sky. The WISE telescope optics will be cooled below 20 degrees Kelvin to keep it colder than the objects in space it will observe so that WISE can see the dim infrared emission from them rather than from the telescope itself. Formulation: Formulation start: 3/03; Preliminary design review: 7/05; Project confirmation: 6/07; Implementation: Project confirmation: 10/06; Launch readiness date 11/09; GAO review: 12/09. Project Essentials: NASA Center Lead: Jet Propulsion Laboratory. International Partner: None. Major Contractors: Ball Aerospace and Technologies Corporation, Space Dynamics Laboratory. Launch Date: December 14, 2009. Launch Location: Vandenberg AFB, Calif. Launch Vehicle: Delta II. Mission Duration: 9 months. Project Performance (then year dollars in millions): Total Project Cost: Baseline Est. (FY 2009): $311.4; Latest (Oct. 2009): $604.6; Change: 0.1%. Formulation Cost: Baseline Est. (FY 2009): $99.5; Latest (Oct. 2009): $99.6; Change: 0.1%. Development Cost: Baseline Est. (FY 2009): $192.1; Latest (Oct. 2009): $194.9; Change: 1.5%. Operations Cost: Baseline Est. (FY 2009): $19.8; Latest (Oct. 2009): $20.0; Change: 1.0%. Launch Schedule: Baseline Est. (FY 2009): 11/2009; Latest (Oct. 2009): 12/2009; Change: 1 month. Project Challenges: * Design Stability; * Funding Issues. Project Status: Wise successfully launched on December 14, 2009 and sent its first images back from space in January 2010. Wide-field Infrared Survey Explorer (WISE): Detailed Project Discussion: WISE project officials identified two mission critical technologies— the solid hydrogen cryostat and the long wavelength infrared detector multiplexer—both of which were assessed as mature at the project's preliminary design review. The solid hydrogen cryostat is a modification of a heritage technology. It is of similar design and construction and manufactured by the same contractor that produced cryostats for previous NASA missions. A project official said the project did not encounter any challenges with the development of the cryostat. The WISE project's design, however, was not stable at the critical design review as the project had released only 70 percent of its engineering drawings. A project official stated that the drawing count and additional analyses, prototypes, and engineering models were used at the critical design review to evaluate the project's design stability. The project has since released the remainder of the engineering drawings. The project did encounter some challenges during testing, which had an impact on the spacecraft's design The Thermal Mass Dynamics Simulator (TMDS), a structural model of the flight cryostat, failed during structural testing. According to a NASA official, analyses done by the project office and the cryostat's contractor did not predict this problem, but finding this problem in the engineering model saved at least a year delay. To mitigate this problem, the project added a soft- ride system to the launch vehicle to reduce loads on the cryostat, which tested successfully in December 2008. The failure also caused the project to accept more project risk by de-scoping two test events in order to regain reserve margin. According to the project office, the remedy cost $2.6 million, but the overall project schedule was not affected. Though the project is currently on track to meet its launch readiness date, the WISE project encountered schedule delays early in its life cycle. According to a project official, the project was not initially confirmed to proceed because of cost and technical concerns. As a result, the official said the project designed a smaller telescope and matured the technology that had concerned the review board. The preliminary design review for WISE was held in July 2005 and the project had its initial confirmation review in November 2005; however, there was a lack of funding in the NASA budget for the WISE project at that time so the formulation phase was extended. At this point in the project, the launch readiness date had slipped from 2008 to June 2009. A second confirmation review was held in November 2006, at which time the launch readiness date was set for November 2009. Although the second confirmation review happened one year later, the launch readiness date set at the original confirmation review only slipped five months since the project was able to make progress during that year. Although WISE contractors completed payload and spacecraft development and successfully integrated the two, the project faced significant risk as cost reserves were depleted during these activities. To avoid the elimination of future testing and staff reductions, an official told us the WISE project received $4 million from NASA Headquarters. WISE successfully launched on December 14, 2009 and is currently in operations. Project Office Comments: The WISE project office provided technical comments to a draft of this assessment, which were incorporated as appropriate. In addition, project officials commented that they believe design has been stable since the Concept Study Review in 2003, which has been a key factor in the excellent cost and schedule performance of the WISE mission. [End of WISE Evaluation] Agency Comments and Our Evaluation: We provided a draft of this report to NASA for review and comment. In its written response, NASA agrees with our findings and states that it will strive to address the challenges that lead to cost and schedule growth in its projects. NASA agrees that GAO's cost and schedule growth figures reflect what the agency has experienced since the baselines were established in response to the 2005 statutory reporting requirements. Importantly, NASA has begun to provide more data regarding cost growth prior to these baselines, and we look forward to working with NASA to increase transparency into cost and schedule information of large-scale projects even further in the future. NASA noted that its projects are high-risk and one-of-a-kind development efforts that do not lend themselves to all the practices of a "business case" approach that we outlined since essential attributes of NASA's project development differ from those of a commercial or production industry. We agree, however NASA could still benefit from a more disciplined approach to its acquisitions whereby decisions are based upon high levels of knowledge. Currently, inherent risks are being exacerbated due to projects moving forward with immature technologies and unstable designs and difficulties working with contractors and international partners, leading to cost and schedule increase which make it hard for the agency to manage its portfolio and make informed investment decisions. NASA's comments are reprinted in appendix I. NASA also provided technical comments, which we addressed throughout the report as appropriate and where sufficient evidence was provided to support significant changes. We will send copies of the report to NASA's Administrator and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI. Signed by: Cristina Chaplain: Director: Acquisition and Sourcing Management: List of Congressional Committees: The Honorable Barbara A. Mikulski: Chairwoman: The Honorable Richard C. Shelby: Ranking Member: Subcommittee on Commerce, Justice, Science, and Related Agencies: Committee on Appropriations: United States Senate: The Honorable Bill Nelson: Chairman: The Honorable David Vitter: Ranking Member: Subcommittee on Science and Space: Committee on Commerce, Science, and Transportation: United States Senate: The Honorable Alan B. Mollohan: Chairman: The Honorable Frank R. Wolf: Ranking Member: Subcommittee on Commerce, Justice, Science, and Related Agencies: Committee on Appropriations: House of Representatives: The Honorable Gabrielle Giffords: Chairwoman: The Honorable Pete Olson: Ranking Member: Subcommittee on Space and Aeronautics: Committee on Science and Technology: House of Representatives: [End of section] Appendix I: Comments from the National Aeronautics and Space Administration: National Aeronautics and Space Administration: Office of the Administrator: Washington, DC 20546-0001: January 15, 2010: Ms. Christina Chaplain: Director, Acquisition and Sourcing Management: United States Government Accountability Office: Washington, DC 20548: Dear Ms. Chaplain: The National Aeronautics and Space Administration (NASA) appreciates the opportunity to comment on the Government Accountability Office (GAO) draft report entitled, "Assessments of Selected Large-Scale Projects" (GAO-10-227SP). NASA values open communications between the NASA and GAO teams on this effort and will continue to strive to work constructively with GAO to identify and address the challenges that may lead to cost and schedule growth of our projects. We are pleased that GAO continues to recognize NASA's ongoing efforts to mitigate acquisition management risk and lay a stronger foundation for reducing project cost and schedule growth. The Agency has recently taken many steps to make progress in this complex area. As was highlighted, NASA began a new initiative, Joint Cost and Schedule Confidence Levels (JCL), which is designed to increase the likelihood of project success at the specified funding level. The application of the JCL process is expected to increase insight by project and program managers and others into uncertainties and contingencies within an integrated cost and schedule plan. Several projects within the GAO assessment have cost and schedule baselines developed under the JCL initiative, specifically the Landsat Data Continuity Mission and the Magnetospheric Multiscale Mission. Further, as NASA expands application of JCL practices, work has also begun on developing a JCL for several more projects within the GAO review, including existing projects such as the James Webb Space Telescope and the Global Precipitation Measurement missions. In conjunction with the JCL initiative, NASA updated, via a NASA interim directive, the NASA Procedural Requirements 7120.5: Space Flight Program and Project Management Requirements. This interim directive strengthens and clarifies the existing program and project management requirements regarding cost and schedule baselining and rebaselining policy. Additionally, NASA will continue to work with GAO to modify and improve our corrective action plan, developed in response to GAO's designation of NASA acquisition management as a high- risk area. In its draft report, GAO states that NASA's project development costs for the 15 projects, in implementation within the review, have increased by an average of over 13 percent from their baseline cost estimates and experienced an average delay of almost 11 months to their launch dates. NASA agrees with the cost and schedule growth figures that are quoted, and they are reflective of what has been experienced since baselines were established in response to the 2005 statutory reporting requirements. The reasons for this growth have been reflected in the resulting reports, as submitted to the Congress. As asserted in the GAO report, this requirement from the Congress has enabled a more consistent reporting among NASA projects and has made the past cost and schedule growth from the original project baseline transparent. Among the projects in the GAO assessment, five out of 15 projects--Gloty, Herschel, NPOESS Preparatory Project, Solar Dynamics Observatory (SDO), and the Stratospheric Observatory for Infrared Astronomy (SOFIA)--had experienced cost growth prior to the baseline provided to Congress. NASA estimates that these five projects experienced an additional $750M of prior cost growth, with more than 80 percent of this figure due to the SOFIA project (estimated at over $600M of prior cost growth), which was originally baselined in fiscal year 1996. The four remaining projects moved from formulation into implementation within two to three years of providing a baseline to Congress; hence, they had not seen significant growth prior to this. If the past cost growth from SOFIA is excluded from the total, the remaining four projects' additional growth constitutes about a two- percent increase above the 13.4 percent average growth figure quoted by GAO. Including all five projects' past growth, it brings the total cost growth to approximately $2.1 B, out of the $66B of estimated life- cycle cost for all 19 projects in the GAO report. While NASA practices many of the elements of GAO's stated "business case" approach, essential attributes of NASA's project development differ from those of a commercial or production entity. For example, unlike a commercial or production entity, NASA's projects are generally high-risk, one-of-a-kind space flight mission developments and rarely enter an "operational" or production state. NASA will continue to work with GAO to adapt the assessment methodology to better reflect NASA's business. NASA will work with GAO to refine the metrics and data collection for the assessment. Further, NASA would like to review and benchmark with the GAO's best practices project data to increase our understanding of when we might improve. NASA agrees that all six of the identified challenges (technical maturity, design stability, contractor performance, development partner performance, funding issues, and launch manifest issues) are experienced in NASA's highly complex, one-of a-kind space flight development efforts. As highlighted last year, NASA identified launch manifest issues as one of the significant causes for project cost and schedule growth, as noted in GAO's analysis. We agree with GAO that the list of six challenges identified in the draft report will evolve as this assessment is continued into the future. In the coming year, NASA will work with GAO on the metrics and data collection for the identified design stability and launch manifest project challenges to account best for the complexity that surrounds these challenges. For example, inherent in the launch manifest issue is the complexity of separating spacecraft development delays from launch vehicle manifest delays. In some cases, a minimal spacecraft readiness slip translates into a disproportionate launch schedule slip, due to the limited flexibility of the expendable launch vehicle manifest (such as was the case with SDO). Other launch delays may occur due to fleet issues such as clearing the Taurus vehicle after the failure of the shroud on the Orbiting Carbon Observatory. Further, the interaction of launch vehicle and spacecraft are not always cleanly separable. For example, once a spacecraft team is aware of a launch vehicle issue, they must act to either adjust the design to address the launch vehicle issue or phase the project spending to maintain critical team skills until the vehicle is ready. These changes to the project will show as a cost growth to the spacecraft even though it may not be driven by the spacecraft development and processing activities. In summary, the set of launch manifest issues presents a project challenge that has many complex aspects which can benefit from refined data collection and metrics. NASA will continue to follow through with the new policies and management attention on cost and schedule growth, in the coming year, with the goal of improving the Agency's performance. We will continue to work with GAO to identify where best to place our efforts. Thank you for the opportunity to comment on this draft report. If you have any questions or require additional information, please contact Julie Pollitt at (202) 3584580. Sincerely, Signed by: Lori B. Garver: Deputy Administrator: [End of section] Appendix II: Objectives, Scope, and Methodology: Our objectives were to report on the status and challenges faced by NASA systems with life-cycle costs of $250 million or more and to discuss broader trends faced by the agency in its management of system acquisitions. In conducting our work, we evaluated performance and identified challenges for each of 19 major projects.[Footnote 24] We summarized our assessments of each individual project in two components—a project profile and a detailed discussion of project challenges. We did not validate the data provided by the National Aeronautics and Space Administration (NASA). However, we took appropriate steps to address data reliability. Specifically, we confirmed the accuracy of NASA- generated data with multiple sources within NASA and, in some cases, with external sources. Additionally, we corroborated data provided to us with published documentation. We determined that the data provided by NASA project offices were sufficiently reliable for our engagement purposes. We developed a standardized data collection instrument (DCI) that was completed by each project office. Through the DCI, we gathered basic information about projects as well as current and projected development activities for those projects. The cost and schedule data estimates that NASA provided were the most recent updates as of October 2009; performance data that NASA provided were the most recent updates as of November 2009. At the time we collected the data, 4 of the 19 projects were in the formulation phase and 15 were in the implementation phase. NASA only provided cost and schedule data for 14 projects in implementation. Despite being in the implementation phase, NASA did not provide cost or schedule data for the Magnetospheric Multiscale (MMS) project. To further understand performance issues, we talked with officials from most project offices and NASA's Office of Program Analysis and Evaluation (PA&E). The results collected from each project office, Mission Directorate, and PA&E were summarized in a 2-page report format providing a project overview; key cost, contract, and schedule data; and a discussion of the challenges associated with the deviation from relevant indicators from best practice standards. The aggregate measures and averages calculated were analyzed for meaningful relationships, e.g. relationship between cost growth and schedule slippage and knowledge maturity attained both at critical milestones and through the various stages of the project life cycle. We identified cost and/or schedule growth as significant where, in either case, a project's cost and/or its schedule exceeded the baselines that trigger reporting to the Congress. To supplement our analysis, we relied on GAO's work over the past years examining acquisition issues across multiple agencies. These reports cover such issues as contracting, program management, acquisition policy, and estimating cost. GAO also has an extensive body of work related to challenges NASA has faced with specific system acquisitions, financial management, and cost estimating. This work provided the context and basis for large parts of the general observations we made about the projects we reviewed. Additionally, the discussions with the individual NASA projects helped us identify further challenges faced by the projects. Together, the past work and additional discussions contributed to our development of a short list of challenges discussed for each project. The challenges we identified and discussed do not represent an exhaustive or exclusive list. They are subject to change and evolution as GAO continues this annual assessment in future years. Our work was performed primarily at NASA headquarters in Washington, D.C. In addition, we visited NASA's Marshall Space Flight Center in Huntsville, Alabama; Dryden Flight Research Center at Edwards Air Force Base in California; and Goddard Space Flight Center in Greenbelt, Maryland, to discuss individual projects. We also met with representatives from NASA's Jet Propulsion Lab in Pasadena, California and a provider of NASA launch services, the United Launch Alliance. Data Limitations: NASA only provided specific cost and schedule estimates for 14 of the 19 projects in our review. For one project, the Magnetospheric Multiscale project, NASA will not formally release its baseline cost and schedule estimates until the fiscal year 2011 budget submission to Congress, and late in our review process agency officials notified us that they will not provide project estimates to GAO until that time. For three of the projects that had not yet entered implementation, NASA provided internal preliminary estimated total (life-cycle) cost ranges and associated schedules, from key decision point B (KDP-B), solely for informational purposes.25 NASA formally establishes cost and schedule baselines, committing itself to cost and schedule targets for a project with a specific and aligned set of planned mission objectives at key decision point C (KDP-C), which follows a non- advocate review (NAR) and preliminary design review (PDR). KDP-C reflects the life-cycle point where NASA approves a project to leave the formulation phase and enter into the implementation phase. NASA explained that preliminary estimates are generated for internal planning and fiscal year budgeting purposes at KDP-B, which occurs mid- stream in the formulation phase, and hence, are not considered a formal commitment by the agency on cost and schedule for the mission deliverables. NASA officials contend that because of changes that occur to a project's scope and technologies between KDP-B and KDP-C, estimates of project cost and schedule can change significantly heading toward KDP-C. Finally, NASA did not provide data for the Global Precipitation Measurement mission because NASA officials said it did not have a requirement for a KDP-B review, because it was authorized to be formulated prior to the requirements of NPR 7120.5D being in place. Project Profile Information on Each Individual Two-Page Assessment: This section of the 2-page assessment outlines the essentials of the project, its cost and schedule performance, and its status. Project essentials reflect pertinent information about each project, including, where applicable, the major contractors and partners involved in the project. These organizations have primary responsibility over a major segment of the project or, in some cases, the entire project. Project performance is depicted according to cost and schedule changes in the various stages of the project life cycle. To assess the cost and schedule changes of each project we obtained data directly from NASA PA&E and from NASA's Integrated Budget and Performance documents. For systems in implementation, we compared the latest available information with baseline cost and schedule estimates set for each project in the fiscal year 2007, 2008, or 2010 budget request. All cost information is presented in nominal "then year" dollars for consistency with budget data.[Footnote 26] Baseline costs are adjusted to reflect the cost accounting structure in NASA's fiscal year 2009 budget estimates. For the fiscal year 2009 budget request, NASA changed its accounting practices from full-cost accounting to reporting only direct costs at the project level. The schedule assessment is based on acquisition cycle time, which is defined as the number of months between the project start, or formulation start, and projected or actual launch date.[Footnote 27] Formulation start generally refers to the initiation of a project; NASA refers to project start as key decision point A, or the beginning of the formulation phase. The preliminary design review typically occurs during the end of the formulation phase, followed by a confirmation review, referred to as key decision point C, which allows the project to move into the implementation phase. The critical design review is held during the final design period of implementation and demonstrates that the maturity of the design is appropriate to support proceeding with full scale fabrication, assembly, integration, and test. Launch readiness is determined through a launch readiness review that verifies that the launch system and spacecraft/payloads are ready for launch. The implementation phase includes the operations of the mission and concludes with project disposal. We assessed the extent to which NASA projects exceeded their cost and schedule baselines. To do this, we compared the project baseline cost and schedule estimates with the current cost and schedule data reported by the project office in October 2009. Project Challenges Discussion on Each Individual Two-Page Assessment: To assess the project challenges for each project, we submitted a data collection instrument to each project office. We also held interviews with representatives from most of the projects to discuss the information on the data collection instrument. These discussions led to identification of further challenges faced by NASA projects. These challenges were largely apparent in the projects that had entered the implementation phase. We then reviewed pertinent project documentation, such as the project plan, schedule, risk assessments, and major project reviews. To assess technology maturity, we asked project officials to assess the technology readiness levels (TRL) of each of the project's critical technologies at various stages of project development. Originally developed by NASA, TRLs are measured on a scale of one to nine, beginning with paper studies of a technology's feasibility and culminating with a technology fully integrated into a completed product. (See appendix IV for the definitions of technology readiness levels.) In most cases, we did not validate the project offices' selection of critical technologies or the determination of the demonstrated level of maturity. However, we sought to clarify the technology readiness levels in those cases where the information provided raised concerns, such as where a critical technology was reported as immature late in the project development cycle. Additionally, we asked project officials to explain the environments in which technologies were tested. Our best practices work has shown that a technology readiness level of 6—-demonstrating a technology as a fully integrated prototype in a relevant environment—-is the level of maturity needed to minimize risks for space systems entering product development. In our assessment, the technologies that have reached technology readiness level 6 are referred to as fully mature because of the difficulty of achieving technology readiness level 7, which is demonstrating maturity in an operational environment—space. Projects with critical technologies that did not achieve maturity by the preliminary design review were assessed as having a technology maturity project challenge. We did not assess technology maturity for those projects which had not yet reached the preliminary design review at the time of this assessment.[Footnote 28] To assess the complexity of heritage technology, we asked project officials to assess the TRL of each of the project's heritage technologies at various stages of project development. We also interviewed project officials about the use of heritage technologies in their projects. We asked them what heritage technologies were being used, what effort was needed to modify the form, fit, and function of the technology for use in the new system, whether the project encountered any problems in modifying the technology, and whether the project considered the heritage technology as a risk to the project. Heritage technologies were not considered critical technologies by several of the projects we reviewed. Based on our interviews, review of data from the data collection instruments, and previous GAO work on space systems, we determined whether complexity of heritage technology was a challenge for a particular project. To assess design stability, we asked project officials to provide the percentage of engineering drawings completed or projected for completion by the preliminary and critical design reviews and as of our current assessment.[Footnote 29] In most cases, we did not verify or validate the percentage of engineering drawings provided by the project office. However, we collected the project offices' rationale for cases where it appeared that only a small number of drawings were completed by the time of the design reviews or where the project office reported significant growth in the number of drawings released after CDR. In accordance with GAO's best practices, projects were assessed as having achieved design stability if they had released at least 90 percent of projected drawings by the critical design review. Projects that had not met this metric were determined to have a design stability project challenge. Though some projects used other methods to assess design stability, such as computer and engineering models and analyses, we did not analyze the use of these other methods and therefore could not assess the design stability of those projects. We could not assess design stability for those projects that had not yet reached the critical design review at the time of this assessment. To assess whether projects encountered challenges with contractor performance, we interviewed project officials about their interaction and experience with contractors. We also relied on interviews we held in 2008 with contractor representatives from Orbital Sciences Corporation, Ball Aerospace and Technologies Corporation, and Raytheon Space Systems about their experiences contracting with NASA. We were informed about contractor performance problems pertaining to their workforce, the supplier base, and technical and corporate experience. We also discussed the use of contract fees with NASA and contractor's representatives. We assessed a project as having this challenge if these contractor performance problems—as confirmed by NASA and, where possible, the project contractor—caused the project to experience a cost overrun, schedule delay, or decrease in mission capability. For projects that did not have a major contractor, we considered this challenge inapplicable to the project. To assess whether projects encountered challenges with development partner performance, we interviewed NASA project officials about their interaction with international or domestic partners during project development. Development partner performance was considered a challenge for the project if project officials indicated that domestic or foreign partners were experiencing problems with project development that impacted the cost, schedule, or performance of the project for NASA. These challenges were specific to the partner organization or caused by a contractor to that partner organization. For projects that did not have an international or domestic development partner, we considered this challenge not applicable to the project. To assess whether projects encountered challenges with funding, we interviewed officials from NASA's Program Analysis and Evaluation Division, NASA project officials, and project contractors about the stability of funding throughout the project life-cycle. Funding stability was considered a challenge if officials indicated that project funding had been interrupted or delayed resulting in an impact to the cost, schedule, or performance of the project, or if project officials indicated that the project budgets do not have sufficient funding in certain years based on the work expected to be accomplished. We corroborated the funding changes and reasons with budget documents when available. To assess whether projects encountered challenges with their launch manifests, we interviewed NASA Launch Services officials and officials from one of NASA's contracted providers for launch services about project launch scheduling, launch windows, and projects that missed their opportunities. Launch manifest was considered a challenge if, after establishing a firm launch date, a project had difficulty rescheduling its launch date because it was not ready, if the project could be affected by another project slipping its launch, or if there were launch vehicle fleet issues. Projects that have not yet entered into the implementation phase have not yet set a firm launch date and were therefore not assessed. In addition, NASA received an appropriation from the American Recovery and Reinvestment Act of 2009 (ARRA). NASA provided a record of projects involved in our review that received ARRA funds. The individual project offices were given an opportunity to comment on and provide technical clarifications to the 2-page assessments prior to their inclusion in the final product. We conducted this performance audit from April 2009 to February 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix III: NASA Life Cycle For Flight Systems Compared to a Knowledge-Based Approach: GAO has previously conducted work on NASA's acquisition policy for spaceflight systems, and in particular, on its alignment with a knowledge-based approach to system acquisitions. The figure below depicts this alignment. Figure 3: NASA's Life Cycle for Flight Systems Compared to a Knowledge- Based Approach: [Refer to PDF for image: illustration] NASA's life cycle for flight systems and ground support projects: Formulation: Pre-phase A: Concept studies; KDP A. Phase A: Concept development; Pre-NAR; SDR; KDP B. Phase B: Preliminary design and technology completion; NAR; PDR; KDP C. Implementation: Phase C: Final design and fabrication; CDR; KDP D. Phase D: System assembly, integration and test, launch; KDP E. Phase E: Operations and sustainment; KDP F. Phase F: Closeout. Knowledge-based approach: Concept and technology development: Knowledge point 1 (KP1): Technologies, time, funding, and other resources match customer needs (Program start). Product development: Integration; Demonstration; Knowledge point 2 (KP2): Design performs as expected. Production: Knowledge point 3 (KP3): Production meets cost, schedule, and quality targets. Management decision reviews: Pre-NAR = preliminary non-advocate review; NAR = non-advocate review; KDP = key decision point. Technical reviews: SDR = system definition review; PDR = preliminary design review; CDR = critical design review. Source: NASA data and GAO analysis. [End of figure] As the figure shows, NASA's policy defines a project life cycle in two phases—the formulation[Footnote 30] and implementation[Footnote 31] phases, which are further divided into incremental pieces: Phase A through Phase E Project formulation consists of Phases A and B, during which time the projects develop and define the project requirements and cost/schedule basis and design for implementation, including an acquisition strategy. During the end of the formulation phase, leading up to the preliminary design review (PDR)[Footnote 32] and non- advocate review (NAR),[Footnote 33] the project team completes its preliminary design and technology development. NASA Interim Directive NM 7120-81 for NASA Procedural Requirements 7120.5D, NASA Space Flight Program and Project Management Requirements, specify that the project complete development of mission-critical or enabling technology, as needed, with demonstrated evidence of required technology qualification (i.e., component and/or breadboard validation in the relevant environment) documented in a technology readiness assessment report. The project must also develop, document, and maintain a project management baseline[Footnote 34] that includes the integrated master schedule and baseline life-cycle cost estimate. Implementing these requirements brings the project closer to ensuring that resources and needs match, but it is not fully consistent with knowledge point 1 of the knowledge-based acquisition life-cycle. Our best practices show that demonstrating technology maturity at this point in the system life cycle should include a system or subsystem model or prototype demonstration in a relevant environment, not only component validation. As written, NASA's policy does not require full technology maturity before a project enters the implementation phase. After project confirmation, the project begins implementation, consisting of phases C, D, E, and F. During phases C and D, the project performs final design and fabrication as well as testing of components and system assembly, integration, test, and launch. Phases E and F consist of operations and sustainment and project closeout. A second design review, the critical design review (CDR),[Footnote 35] is held during the implementation phase toward the end of phase C. The purpose of the CDR is to demonstrate that the maturity of the design is appropriate to support proceeding with full scale fabrication, assembly, integration, and test. Though this review is not a formal decision review, its requirements for a mature design and ability to meet mission performance requirements within the identified cost and schedule constraints are similar to knowledge expected at knowledge point 2 of the knowledge-based acquisition life-cycle. Furthermore, after CDR, the project must be approved at EDP D before continuing into the next phase. The NASA acquisition life-cycle lacks a major decision review at knowledge point 3 to demonstrate that production processes are mature. According to NASA officials, the agency rarely enters a formal production phase due to the small quantities of space systems that they build. [End of section] Appendix IV: Technology Readiness Levels: Technology readiness level: 1. Basic principles observed and reported. Description: Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology's basic properties; Hardware: None (paper studies and analysis); Demonstration environment: None. Technology readiness level: 2. Technology concept and/or application formulated. Description: Invention begins. Once basic principles are observed, practical applications can be invented. The application is speculative and there is no proof or detailed analysis to support the assumption. Examples are still limited to paper studies. Hardware: None (paper studies and None analysis); Demonstration environment: None. Technology readiness level: 3. Analytical and experimental critical function and/or characteristic proof of concept. Description: Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative. Hardware: Analytical studies and demonstration of nonscale individual components (pieces of subsystem); Demonstration environment: Lab. Technology readiness level: 4. Component and/or breadboard. Validation in laboratory environment. Description: Basic technological components are integrated to establish that the pieces will work together. This is relatively "low fidelity' compared to the eventual system. Examples include integration of "ad hoc" hardware in a laboratory. Hardware: Low fidelity breadboard. Integration of nonscale components to show pieces will work together. Not fully functional or form or fit but representative of technically feasible approach suitable for flight articles. Demonstration environment: Lab. Technology readiness level: 5. Component and/or breadboard validation in relevant environment. Description: Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so that the technology can be tested in a simulated environment. Examples include "high fidelity" laboratory integration of components. Hardware: High fidelity breadboard. Functionally equivalent but not necessarily form and/or fit (size weight, materials, etc). Should be approaching appropriate scale. May include integration of several components with reasonably realistic support elements/subsystems to demonstrate functionality. Demonstration environment: Lab demonstrating functionality but not form and fit. May include flight demonstrating breadboard in surrogate aircraft. Technology ready for detailed design studies. Technology readiness level: 6. System/subsystem model or prototype demonstration in a relevant environment. Description: Representative model or prototype system, which is well beyond the breadboard tested for TRL 5, is tested in a relevant environment. Represents a major step up in a technology’s demonstrated readiness. Examples include testing a prototype in a high fidelity laboratory environment or in simulated realistic environment. Hardware: Prototype. Should be very close to form, fit and function. Probably includes the integration of many new components and realistic supporting elements/subsystems if needed to demonstrate full functionality of the subsystem. Demonstration environment: High-fidelity lab demonstration or limited/restricted flight demonstration for a relevant environment. Integration of technology is well defined. Technology readiness level: 7. System prototype demonstration in a realistic environment. Description: Prototype near or at planned operational system. Represents a major step up from TRL 6, requiring the demonstration of an actual system prototype in a realistic environment, such as in an aircraft, vehicle or space. Examples include testing the prototype in a test bed aircraft. Hardware: Prototype. Should be form, fit and function integrated with other key supporting elements/subsystems to demonstrate full functionality of subsystem. Demonstration environment: Flight demonstration in representative realistic environment such as flying test bed or demonstrator aircraft. Technology is well substantiated with test data. Technology readiness level: 8. Actual system completed and “flight qualified” through test and demonstration. Description: Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications. Hardware: Flight qualified hardware. Demonstration environment: Development Test and Evaluation (DT&E) in the actual system application. Technology readiness level: 9. Actual system “flight proven” through successful mission operations. Description: Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. In almost all cases, this is the end of the last "bug fixing" aspects of true system development. Examples include using the system under operational mission conditions. Hardware: Actual system in final form. Demonstration environment: Operational Test and Evaluation (OT&E) in operational mission conditions. Source: GAO and its analysis of National Aeronautics and Space Administration data. [End of table] [End of section] Appendix V: NASA Projects Receiving Additional Funding: There are 7 NASA projects in our review, including all of those in formulation, that are receiving money from the American Recovery and Reinvestment Act (ARRA) of 2009.[Footnote 36] See table 4 below for the NASA projects in our review receiving this funding and the intended use of those funds. Table 4: ARRA Funding for Reviewed NASA Projects: Project: Aquarius; ARRA funds: $15.6 million; NASA's Intended use of funds: * Maintain the current workforce through the launch. Project: Ares I; ARRA funds: $108.9 million; NASA's Intended use of funds: * Manufacture and assemble multiple Ares J2-X engine components and subsystems for development testing. * Ares Upper Stage Vertical Assembly Tool materials, test equipment, engineering analysis and manufacturing process development at Marshall Space Flight Center. * Completion of the Ares A-3 Test Stand and preparation for test operations. Project: Glory; ARRA funds: $21.0 million; NASA's Intended use of funds: * Maintain the current contractor workforce through the launch. Project: GPM; ARRA funds: $32.0 million; NASA's Intended use of funds: * Accelerate construction of the GPM Microwave Imager (GMI) instruments to ensure successful launch of the mission at the earliest possible opportunity. Project: JWST; ARRA funds: $75.0 million; NASA's Intended use of funds: * Spacecraft development activities including design and fabrication of key component systems to increase the likelihood that JWST will launch on the planned launch date. Project: LDCM; ARRA funds: $51.6 million; NASA's Intended use of funds: * Initiate development of the TIRS and integration of the instrument onto the spacecraft. Project: Orion; ARRA funds: $165.9 million; NASA's Intended use of funds: * Reduce schedule risk by initiating purchases of long lead components and moving forward with the design of Orion Engineering Development Units. * Reduce technical risk by adding additional fidelity to the Orion Ground Test Articles. * Technology development testing to improve the crew safety of the Orion spacecraft. Source: GAO presentation of data provided by NASA. [End of section] Appendix VI: GAO Contact and Staff Acknowledgments: GAO Contact: Cristina Chaplain (202) 512-4841 or chaplainc@gao.gov. Acknowledgments: In addition to the contact named above, Jim Morrison, Assistant Director; Jessica M. Berkholtz; Greg Campbell; Richard A. Cederholm; Kristine R. Hassinger, Jeff R. Jensen; Kenneth E. Patton; Brian A. Tittle; and Letisha T. Watson made key contributions to this report. [End of section] Footnotes: [1] GAO, NASA: Assessments of Selected Large-Scale Projects, [hyperlink, http://www.gao.gov/products/GAO-09-306SP] (Washington, D.C.: Mar. 2, 2009). [2] If development cost of a program will exceed the baseline estimate by more than 30 percent, then NASA is required to seek reauthorization from Congress in order to continue the program. If the program is reauthorized, NASA is required to establish new cost and schedule baselines. 42 U.S.C. § 16613(e). [3] National Aeronautics and Space Administration Authorization Act of 2005, Pub. L. No. 109161, §103; 42 U.S.C. § 16613(b). [4] 42 U.S.C. § 16613(d). [5] Pub. L. No. 111-5, 123 Stat. 115, Division A, Title II (2009). [6] NASA did not provide cost or schedule data for the Magnetospheric Multiscale (MMS) project despite that project being in the implementation phase. NASA did provide preliminary estimates in the form of cost ranges for three of the four projects in the formulation phase. Since the values provided were ranges, rather than specific values, we did not include these projects in our analysis. Further, the agency did not provide schedule baselines for these projects so we could not determine any schedule changes they experienced. [7] NASA is required to report to Congress if development cost of a program is likely to exceed the baseline estimate by 15 percent or more, or if a milestone is likely to be delayed by 6 months or more. 42 U.S.C. § 16613(d). [8] GAO, Defense Acquisitions: Key Decisions to Be Made on Future Combat System, [hyperlink, http://www.gao.gov/products/GAO-07-376] (Washington, D.C.: Mar. 15, 2007); Defense Acquisitions: Improved Business Case Key for Future Combat System's Success, [hyperlink, http://www.gao.gov/products/GAO-06-564T] (Washington, D.C.: Apr. 4, 2006); NASA: Implementing a Knowledge-Based Acquisition Framework Could Lead to Better Investment Decisions and Project Outcomes, [hyperlink, http://www.gao.gov/products/GAO-06-218] (Washington, D.C.: Dec. 21, 2005); NASA's Space Vision: Business Case for Prometheus 1 Needed to Ensure Requirements Match Available Resources, [hyperlink, http://www.gao.gov/products/GAO-05-242] (Washington, D.C.: Feb. 28, 2005). [9] [hyperlink, http://www.gao.gov/products/GAO-05-242]. [10] The revised policy, issued March 6, 2007, is NASA Procedural Requirements 7120.5D, NASA Spaceflight Program and Project Management Requirements (Mar. 6, 2007). On September 22, 2009, NASA Interim Directive (NID) NM 7120-81 for NASA Procedural Requirements (NPR) 7120.5D was issued, hereinafter cited as MD for NPR 7120.5D (Sept. 22, 2009). [11] MD for NPR 7120.5D, paragraph 2.4.2 (Sept. 22, 2009). [12] GAO, High-Risk Series: An Update, [hyperlink, http://www.gao.gov/products/GAO-07-310] (Washington, D.C.: Jan. 2007). [13] National Aeronautics and Space Administration, Plan for Improvement in the GAO High-Risk Area of Contract Management (Oct. 31, 2007). [14] For purposes of our analysis, cost or schedule growth is significant if it exceeds the baseline thresholds that trigger reporting to Congress under the law. The thresholds are development cost growth of 15 percent or more from the baseline cost estimate or a milestone delay of 6 months or more beyond the baseline schedule estimate. 42 U.S.C. § 16613(d). [15] 42 U.S.C. § 16613(b). [16] If development cost of a program will exceed the baseline estimate by more than 30 percent, then NASA is required to seek reauthorization from Congress in order to continue the program. If the program is reauthorized, NASA is required to establish new cost and schedule baselines. 42 U.S.C. § 16613(e). [17] 42 U.S.C. § 16613(e). [18] The "product development" stage on GAO's knowledge-based approach is equivalent to "implementation" on NASA's life cycle, see appendix III. [19] NASA Procedural Requirements (NPR) 7123.1A, NASA Systems Engineering Processes and Requirements, Appendix G, paragraph G.19(b) (Mar. 26, 2007). [20] Appendix IV provides a description of the metrics used to assess technology maturity. [21] Projects will modify the form, fit, and function of a heritage technology to adapt to the new environment. For example, the size or the weight of the component may change or the technology may function differently than its use in a previous mission. [22] NASA reported to the Congress that these projects had exceeded their cost and/or schedule baselines. 42 U.S.C. § 16613(d). [23] United Launch Alliance is a provider of launch services to the U.S. Government. [24] We originally collected information on 21 projects, but two missions—Dawn and the Gamma-ray Large Area Space Telescope—were later excluded since they were both in continuing operations and development teams were no longer available to be interviewed. [25] These missions include Ares I, the Landsat Data Continuity Mission, and Orion. [26] Because of changes in NASA's accounting structure, its historical cost data are relatively inconsistent. As such, we used "then-year" dollars to report data consistent with the data NASA reported to us. [27] Some projects reported that their spacecraft would be ready for launch sooner than the date that the launch authority could provide actual launch services. In these cases, we used the actual launch date for our analysis rather than the date that the project reported readiness. [28] According to NASA officials, projects that were in formulation at the time of the agency's 2007 revision of its project management policy are required to comply with that policy. Projects that had already entered implementation at the time of the revision were directed to implement those requirements that would not adversely affect the project's cost and schedule baselines. [29] In our calculation for percentage of total number of drawings projected for release, we used the number of drawings released at critical design review as a fraction of the total number of drawings projected, including where a growth in drawings occurred. So, the denominator in the calculation may have been larger than what was projected at the critical design review. We believe that this more accurately reflected the design stability of the project. [30] NASA defines formulation as the identification of how the program or project supports the agency's strategic needs, goals, and objectives; the assessment of feasibility, technology and concepts; risk assessment, team building, development of operations concepts and acquisition strategies; establishment of high-level requirements and success criteria; the preparation of plans, budgets, and schedules essential to the success of a program or project; and the establishment of control systems to ensure performance to those plans and alignment with current agency strategies. NID for NPR 7120.5D, paragraph 1.2.1(a) (Sept. 22, 2009). [31] The implementation phase is defined as the execution of approved plans for the development and operation of the program/project, and the use of control systems to ensure performance to approved plans and continued alignment with the agency's strategic needs, goals, and objectives. NID for NPR 7120.5D, paragraph 1.2.1(c) (Sept. 22, 2009). [32] According to MD for NPR 7120.5D, Table 2-7 (Sept. 22, 2009), the PDR demonstrates that the preliminary design meets all system requirements with acceptable risk and within the cost and schedule constraints and establishes the basis for proceeding with detailed design. It shows that the correct design option has been selected, interfaces have been identified, and verification methods have been described. Full baseline cost and schedules, as well as risk assessments, management systems, and metrics are presented. [33] According to MD for NPR 7120.5D, Appendix A (Sept. 22, 2009), a non-advocate review (NAR) is comprised of the analysis of a proposed program or project by a (non-advocate) team composed of management, technical, and resources experts (personnel) from outside the advocacy chain of the proposed program or project. It provides agency management with an independent assessment of the readiness of the program/project to proceed into implementation. [34] The management baseline is the integrated set of requirements, cost, schedule, technical content, and associated joint confidence level that forms the foundation for program or project execution and reporting done as part of NASAs performance assessment and governance process. MD for NPR 7120.5D, paragraph 2.1.8.2 and Appendix A (Sept. 22, 2009). [35] According to MD for NPR 7120.5D, Table 2-7 (Sept. 22, 2009), the CDR demonstrates that the maturity of the design is appropriate to support proceeding with full scale fabrication, assembly, integration, and test, and that the technical effort is on track to complete the flight and ground system development and mission operations in order to meet mission performance requirements within the identified cost and schedule constraints. Progress against management plans, budget, and schedule, as well as risk assessments are presented. [36] Pub. L. No. 111-5, 123 Stat. 115, Division A, Title II (2009). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: