This is the accessible text file for GAO report number GAO-09-436T entitled 'NASA: Projects Need More Disciplined Oversight and Management to Address Key Challenges' which was released on March 5, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on Space and Aeronautics, Committee on Science and Technology, House of Representatives: United States Government Accountability Office: GAO: For Release on Delivery: Expected at 10:00 a.m. EST: Thursday, March 5, 2009: NASA: Projects Need More Disciplined Oversight and Management to Address Key Challenges: Statement of Cristina Chaplain, Director: Acquisition and Sourcing Management: GAO-09-436T: Madam Chairwoman and Members of the Subcommittee: Thank you for inviting me to discuss the National Aeronautics and Space Administration's (NASA) oversight and management of its major projects. As you know, in 1990, GAO designated NASA's contract management as high risk in view of persistent cost growth and schedule slippage in the majority of its major projects. Since that time, GAO's high-risk work has focused on identifying a number of causal factors, including antiquated financial management systems, poor cost estimating, and undefinitized contracts. Because cost growth and schedule delays persist, this area - now titled acquisition management because of the scope of issues that need to be resolved - remains high risk. To its credit, NASA has recently made a concerted effort to improve its acquisition management. In 2007, NASA developed a comprehensive plan to address systemic weaknesses related to how it manages its acquisitions. The plan specifically seeks to strengthen program/project management, increase accuracy in cost estimating, facilitate monitoring of contractor cost performance, improve agency wide business processes, and improve financial management. While we applaud these efforts our recent work has shown that NASA needs to pay more attention to effective project management. It needs to adopt best practices that focus on closing gaps in knowledge about requirements, technologies, funding, time and other resources before it makes commitments to large-scale programs. For instance, the Mars Science Laboratory, which was already over budget, recently announced a 2-year launch delay. Current estimates suggest that the price of this delay may be $400 million--which drives the current project life-cycle cost estimate to $2.3 billion; up from its initial confirmation estimate of $1.6 billion. Also, in just one year, the development costs of NASA's Glory mission increased by 54 percent, or almost $100 million, because of problems NASA's contractor is having developing a key sensor. Total project costs for another project, Kepler have increased almost another $100 million within 2 fiscal years because of similar issues. Taken together, these and other unanticipated cost increases hamper NASA's ability to fund new projects, continue existing ones, and pave the way to a post-shuttle space exploration environment. Given the constrained fiscal environment and pressure on discretionary spending it is critical that NASA get the most out of its investment dollars for its space systems. The agency is increasingly being asked to expand its portfolio to support important scientific missions including the study of climate change. Therefore, it is exceedingly important that these resources be managed as effectively and efficiently as possible for success. The recent launch failure of the Orbiting Carbon Observatory is an all-too-grim reminder of how much time, hard work, and resources can be for naught when a space project cannot execute its mission. In response to congressional direction, we have prepared a comprehensive report on the management and oversight of NASA's major projects. It contains summaries of 18 projects with a combined life- cycle cost exceeding $50 billion. It also contains an assessment of issues affecting projects across-the-board. A copy of this report is now available on GAO's Website [hyperlink, http://www.gao.gov]. [Footnote 1] In conducting this work, we compared projects against best practice criteria for system development including attainment of knowledge on technologies and design, as well as various aspects of program management. We expect to continue this assessment on an annual basis and to continually refine our examination so that our work can inform your oversight and NASA's own efforts to improve in the high risk area of acquisition management. In responding to our report, NASA asserted that the unique nature of its work and external factors beyond its control make it difficult to apply the same criteria that we apply to other major government acquisitions, particularly those with large production runs. We disagree. The criteria we used to assess NASA's projects represent commonly accepted, fundamental tenets of disciplined project management, regardless of complexity or quantity. In fact, the concept of the knowledge-based approach we use has been adopted in NASA's own acquisition policy. Key criteria that we use have been developed by NASA and/or incorporated into its engineering policy. Moreover, facing long-standing cost and schedule growth and performance shortfalls, the Department of Defense (DOD) acknowledges the need for a knowledge based approach in the Air Force's "back to basics" policy for space systems. Lastly, we remain open to discussions with NASA as to whether additional criteria can and should be applied to its systems to ensure that decisions to move forward in development are well-informed and ultimately, that taxpayer dollars are well spent. Today I will be highlighting the results of this work, the actions NASA is taking to address the concerns raised in our high risk report and better position its projects to meet their goals, and what we believe is necessary to make these actions successful. Because we also have responsibility for examining military space systems, we will also highlight common challenges with space acquisitions within NASA and the Department of Defense (DOD). This testimony is based on previously issued GAO work, which was conducted in accordance with generally accepted government auditing standards. Acquisition Management Problems Persist: We assessed 18 projects in NASA's current portfolio. Four were in the "formulation" phase, a time when system concepts and technologies are still being explored and 14 were in the "implementation" phase[Footnote 2], where system design is completed, scientific instruments are integrated, and a spacecraft is fabricated. When implementation begins, it is expected that project officials know enough about a project's requirements and what resources are necessary to meet those requirements that they can reliably predict the cost and schedule necessary to achieve its goals. Reaching this point requires investment. In some cases, projects that we reviewed spent 2 to 5 years and up to $100 million or more before being able to formally set cost and schedule estimates. Ten of the projects in our assessment for which we received data and that had entered the implementation phase experienced significant cost and/or schedule growth from their project baselines.[Footnote 3] Based on our analysis, development costs for projects in our review increased by an average of almost 13 percent from their baseline cost estimates- -all in just two or three years--including one that went up more than 50 percent. It should be noted that a number of these projects had experienced considerably more cost growth before a baseline was established in response to statutory reporting requirements. Our analysis also shows that projects in our review had an average delay of 11 months to their launch dates. We found challenges in five areas that occurred throughout the various projects we reviewed that can contribute to project cost and schedule growth. These are not necessarily unique to NASA projects and many have been identified in other weapon and space systems that we have reviewed and have been prevalent in the agency for decades. * Technology maturity. Four of the 13 projects in our assessment for which we received data and that had entered the implementation phase did so without first maturing all critical technologies, that is they did not know that technologies central to the project's success could work as intended before beginning the process of fabricating the spacecraft. This means that knowledge needed to make these technologies work remained unknown well into development. Consequences accrue to projects that are still working to mature technologies well into system development, when they should be focusing on maturing system design and preparing for production. Simply put, projects that start with mature technologies experience less cost growth than those that start with immature technologies. * Design stability. The majority of the projects in our assessment that held a critical design review did so without first achieving a stable design. If design stability is not achieved, but product development continues, costly re-designs to address changes to project requirements and unforeseen challenges can occur. All of the projects in our assessment that had reached their critical design review and that provided data on engineering drawings experienced some growth in the total number of design drawings after their critical design review. Growth ranged from 8 percent to, in the case of two projects, well over 100 percent. Some of this increase can be attributed to change in system design after critical design review. * Complexity of heritage technology. More than half the projects in the implementation phase--eight of them--encountered challenges in integrating or modifying heritage technologies. Additionally, two projects in formulation--Ares I and Orion--also encountered this problem. We found that the projects that relied on heritage technologies underestimated the effort required to modify them to the necessary form, fit, or function. * Contractor performance. Six of the seven projects that cited contractor performance as a challenge also experienced significant cost and/or schedule growth. For example, through our discussions with the project offices, we were informed that contractors encountered technical and design problems with hardware that disrupted development progress. * Development partner performance. Five of the thirteen projects we reviewed encountered challenges with a development partner. In these cases, the development partner could not meet its commitments to the project within planned timeframes. This may have been a result of problems within the specific development partner organization or as a result of problems faced by a contractor to that development partner. Common Acquisition Management Challenges Persist between NASA and DOD: The challenges we identified in the NASA assessment are similar to ones we have identified in other weapon systems, including Defense space systems. We testified last year that DOD space system cost growth was attributable to programs starting before they have assurance that capabilities being pursued can be achieved within available resources and time constraints. For example, DOD's National Polar-orbiting Operational Environmental Satellite System (NPOESS) has increased in cost from roughly $6 billion to over $11 billion because of challenges with maturing key technologies. We have also tied acquisition problems in space systems to inadequate contracting strategies and contract and program management weaknesses. Further, we issued a report in 2006 that found DOD space system cost estimates were consistently optimistic. For example, DOD's Space-Based Infrared High program was originally expected to cost about $4 billion and is now expected to cost more than $10 billion. We have found these problems are largely rooted in the failure to match the customer's needs with the developer's resources--technical knowledge, timing, and funding--when starting product development. In other words, commitments were made to achieving certain capabilities without knowing whether technologies and/or designs being pursued could really work as intended. Time and costs were consistently underestimated. As we have discussed in previous work on space systems at both DOD and NASA, a knowledge-based approach to acquisitions, regardless of the uniqueness or complexity of the system is beneficial because it allows program managers the opportunity to gain enough knowledge to identify potential challenges earlier in development and make more realistic assumptions about what they can achieve. NASA Is Making a Concerted Effort to Reduce High Risk in Acquisition Management but More Needs to Be Done: NASA has also taken significant steps to improve in the high risk area of acquisition management. For example, NASA revised its acquisition and engineering polices to incorporate elements of a knowledge-based approach that should allow the agency to make informed decisions. The agency is also instituting a new approach whereby senior leadership is reviewing acquisition strategies earlier in the process and has developed broad procurement tenets to guide the agency's procurement practices. Further, NASA is working to improve management oversight of project cost, schedule, and technical performance with the establishment of a baseline performance review with senior management. In order to improve it's contracting and procurement process, NASA has instituted an agency wide standard contract-writing application intended to ensure all contracts include the most up-to-date NASA contract clauses and to improve the efficiency of the contracting process. NASA is also requiring project managers to quantify the program risks they identify and collect more consistent data on project cost and technologies. It is taking other actions to enhance cost estimating methodologies and to ensure that independent estimates are used. These changes brought the policy more in line with best practices for product development. However, as we previously reported,[Footnote 4] NASA lacks defined requirements across centers and mission directorates for consistent metrics that demonstrate knowledge attainment through the development cycle. In order for a disciplined approach to take hold, we would expect project officials across the agency to be held accountable for following the same required policies. More steps also need to be taken to manage risk factors that NASA believes are outside of its control. NASA asserts that contractor deficiencies, launch manifest issues, partner performance, and funding instability are to blame for the significant cost and schedule growth on many of its projects that we reviewed. Such unforeseen events, however, should be addressed in project-level, budgeting and resource planning through the development of adequate levels of contingency funds. NASA cannot be expected to predict unforeseen challenges, but being disciplined while managing resources, conducting active oversight of contractors, and working closely with partners can put projects in a better position to mitigate these risks should they occur. Realistically planning for and retiring technical or engineering risks early in product development allows the project to target reserves to issues NASA believes are outside of its control. In conclusion, managing resources as effectively and efficiently as possible is more important than ever for NASA. The agency is undertaking a new multi-billion dollar program to develop the next generation of spacecraft for human spaceflight and at a time when it is faced with increasing demands to support important scientific missions, including the study of climate change, and to increase aeronautics research and development. By allowing major investment commitments to continue to be made with unknowns about technology and design readiness, contractor capabilities, requirements, and/or funding, NASA will merely be exacerbating the inherent risks it already faces in developing and delivering new space systems. Programs will likely continue to experience problems that require more time and money to address than anticipated. Over the long run, the extra investment required to address these problems may well prevent NASA from pursuing more critical science and space exploration missions. By contrast, by continuing to implement its acquisition management reforms and ensuring programs do not move forward with such unknowns, NASA can better align customer expectations with resources, minimize problems that could hurt programs, and maximize it ability to meet increased demands. Madam Chairwoman, this concludes my statement. I will be happy to answer any questions that you have. GAO Contacts and Staff Acknowledgments: For additional information, please contact Cristina Chaplain at 202- 512-4841 or chaplainc@gao.gov. Individuals making contributions to this testimony include Jim Morrison, Assistant Director; Shelby S. Oakley, Assistant Director; Greg Campbell; Richard A. Cederholm; Brendan S. Culley; Deanna R. Laufer; Kenneth E. Patton; and Letisha T. Watson. [End of section] Footnotes: [1] We only received data for 13 of the 14 projects in implementation. NASA did not provide cost or schedule data for the James Webb Space Telescope, which is in implementation. [2] For purposes of our analysis, significant cost and schedule growth occurs when a project's cost and/or its schedule growth exceeds the thresholds established for Congressional reporting per the National Aeronautics and Space Administration Authorization Act of 2005, Pub. L. No. 109-161, §103; 42 U.S.C. §16613 (b), (f) (4). [3] GAO, NASA: Sound Management and Oversight Key to Addressing Crew Exploration Vehicle Project Risks. [hyperlink, http://www.gao.gov/products/GAO-06-1127T] (Washington, D.C.: Sep. 28, 2006). [4] GAO, NASA: Assessments of Selected Large-Scale Projects. [hyperlink, http://www.gao.gov/products/GAO-09-306SP] (Washington, D.C.: Mar. 2, 2009). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: