James Webb Space Telescope:

Actions Needed to Improve Cost Estimate and Oversight of Test and Integration

GAO-13-4: Published: Dec 3, 2012. Publicly Released: Dec 3, 2012.

Additional Materials:

Contact:

Cristina T. Chaplain
(202) 512-4841
chaplainc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Aeronautics and Space Administration (NASA) has provided significantly more time and money to the James Webb Space Telescope (JWST) than previously planned and expressed high confidence in the project's new baselines. Its current cost estimate reflects some features of best practices for developing reliable and credible estimates. For example, the estimate substantially meets one of four cost characteristics--comprehensive--that GAO looks for in a reliable cost estimate, in part because all life cycle costs were included. The estimate, however, only partially met the other three characteristics--well documented, accurate, and credible--which detracts from its reliability. For example, the estimate's accuracy, and therefore the confidence level assigned to the estimate, was lessened by the summary schedule used for the joint cost and schedule risk analysis because it did not provide enough detail to determine how risks were applied to critical project activities. The estimate's credibility was also lessened because officials did not perform a sensitivity analysis that would have identified key drivers of costs, such as workforce size. Program officials believe that it would have been difficult to fully address all best practice characteristics. GAO believes there is time to improve the estimate and enhance the prospects for delivering the project according to plan.

Project officials report that the JWST schedule has 14 months of reserve, which meets Goddard guidance for schedule reserve; however, only 7 of the 14 months are likely to be available for the last three of JWST's five complex integration and test efforts. GAO's prior work shows that the integration and test phases are where problems are commonly found and schedules tend to slip. Given that JWST has a challenging integration and test schedule, this could particularly be likely. The project has made some significant progress in the past year, notably successfully completing development of the 18 primary mirror segments--considered JWST's top technical risk. Nevertheless, ongoing challenges are indicative of the kinds of issues that can require significant effort to address. For example, instrument challenges have delayed the first integration and test effort. In addition, key long-term risks on subsystems with a significant amount of work remaining will not be retired until 2016. Currently, NASA's plan for project oversight calls for one independent mission-level system integration review about 13 months before launch. While this is consistent with what NASA requires for its projects, this approach may not be sufficient for a project as complex as JWST.

JWST has taken several steps to improve communications and oversight of the project and its contractors--such as taking over responsibility for mission systems engineering from the prime contractor; instituting meetings that include various levels of NASA, contractor, and subcontractor management; and implementing a new risk management system to allow for better tracking of risks. The enhancements to the oversight of the project are steps in the right direction, but it will take time to assess their effectiveness and ensure that the efforts are sustained by the project in the future. Reductions in travel budgets, however, could require the project to adjust the oversight approach that was adopted as a result of the replan. Additional reductions in travel budgets are anticipated in future years, but officials do not have a plan to address such reductions and their potential impact on continuing the current oversight approach.

Why GAO Did This Study

JWST is one of NASA's most expensive and technologically advanced science projects, intended to advance understanding of the origin of the universe. In 2011, JWST was rebaselined with a life cycle cost estimate of $8.8 billion and a launch readiness date in October 2018--almost nine times the cost and more than a decade later than originally projected in 1999. Concern about the magnitude of JWST's cost increase and schedule delay and their effects on NASA's progress on other high-priority missions led conferees for the Consolidated and Further Continuing Appropriations Act, 2012, to direct GAO to report on the project. Specifically, GAO assessed (1) the extent to which NASA's revised cost and schedule estimates are reliable based on best practices, (2) the major risks and technological challenges JWST faces, and (3) the extent to which NASA has improved oversight of JWST. To do this, GAO compared NASA's revised cost and schedule estimates with best practice criteria, reviewed relevant contractor and NASA documents, and interviewed project and contractor officials.

What GAO Recommends

GAO recommends NASA take six actions including, among others, to take steps to improve its cost estimate; to conduct an additional, earlier independent review of test and integration activities; and to develop a long-term oversight plan that anticipates planned travel budget reductions.

For more information, contact Cristina Chaplain at (202) 512-4841 or chaplainc@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to conduct a separate independent review prior to the beginning of the Optical Telescope Element and Integrated Science Instrument Module and spacecraft integration and test efforts to allow the project's independent standing review board the opportunity to evaluate the readiness of the project to move forward, given the lack of schedule flexibility once these efforts are under way.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: The JWST project has now scheduled an independent systems integration review for Optical Telescope and Integrated Science Instrument Module (OTIS) in May 2016, prior to the beginning of the OTIS integration and test effort that same month. In addition, the JWST project has scheduled the spacecraft element (SCE) readiness review in March 2016, prior to the beginning of the spacecraft integration and test effort currently scheduled to begin in April 2016.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to improve cost estimate documentation and continually update it to reflect earned value management actual costs and record any reasons for variances.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: The agency has taken steps toward implementation of this recommendation by performing analysis of earned value management data for most contracted and internally performed work, but not all. The agency has plans to develop an analysis tool for work being performed by a non-profit organization in development of the JWST ground systems. However, there is no indication that the agency updates the cost estimate with this information.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to conduct a sensitivity analysis on the number of staff working on the program to determine how staff variations affect the cost estimate.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: In commenting on this recommendation, the agency partially concurred with this recommendation, citing it met the intent, but the agency has not taken actions to implement this recommendation. For example, NASA cites that it took a number of steps to determine how staffing variations affect its cost estimate and to ensure the project has sufficient margin to address changes in workforce requirements. The analysis performed, however, was a part of the cost/schedule risk assessment that was based on information at too high a level of detail to sufficiently account for risk.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to perform an updated integrated cost/schedule risk analysis, or joint cost and schedule confidence level analysis, using a schedule that meets best practices and includes enough detail so that risks can be appropriately mapped to activities and costs; historical, analogous data should be used to support the risk analysis.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: In commenting on this recommendation, the agency concurred but has not taken actions to implement this recommendation. NASA stated that it performs monthly integrated programmatic and cost/schedule risk analysis. For example, the JWST project identifies risks, assesses encumbrances to its cost reserves, tracks actual costs against planned costs, and performs earned value management and schedule analyses. However, there is no integrated assessment of this information performed that would be in line with cost and schedule risk analysis best practices.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to conduct a separate independent review prior to the beginning of the Optical Telescope Element and Integrated ScienceInstrument Module and spacecraft integration and test efforts to allow the project's independent standing review board the opportunity to evaluate the readiness of the project to move forward, given the lack of schedule flexibility once these efforts are under way.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to schedule the management review and approval to proceed to integration and test (key decision point D or KDP-D) prior to the start of observatory integration and test effort.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: In commenting on this recommendation, the agency partially concurred but has not taken actions to implement this recommendation. The agency cites that the Standing Review Board's outbrief of the results of the System Integration Review to the NASA Associate Administrator prior to the start of observatory integration and test meets the intent of this recommendation. NASA policy, however, clearly states that the key decision point and approval by the NASA Associate Administrator should be held prior to integration and test activities commence. The JWST project has this management review and approval 2 months after the start of observatory integration and test.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to devise an effective, long-term plan for project office oversight of its contractors that takes into consideration the anticipated travel budget reductions.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: In commenting on this recommendation, the agency concurred but has not taken actions to implement this recommendation. The JWST project reports that it did not incur any reductions to its travel budget in fiscal year 2013. While this may be the case, future year budgets may still be constrained. The project should devise an effective, long-term plan for oversight of its contractors in light of this uncertain budget environment.

    Jul 23, 2014

    Jul 16, 2014

    Jun 20, 2014

    May 15, 2014

    May 8, 2014

    Apr 15, 2014

    Mar 12, 2014

    Feb 4, 2014

    Jan 13, 2014

    Jan 8, 2014

    Looking for more? Browse all our products here