James Webb Space Telescope:

Actions Needed to Improve Cost Estimate and Oversight of Test and Integration

GAO-13-4: Published: Dec 3, 2012. Publicly Released: Dec 3, 2012.

Additional Materials:

Contact:

Cristina T. Chaplain
(202) 512-4841
chaplainc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Aeronautics and Space Administration (NASA) has provided significantly more time and money to the James Webb Space Telescope (JWST) than previously planned and expressed high confidence in the project's new baselines. Its current cost estimate reflects some features of best practices for developing reliable and credible estimates. For example, the estimate substantially meets one of four cost characteristics--comprehensive--that GAO looks for in a reliable cost estimate, in part because all life cycle costs were included. The estimate, however, only partially met the other three characteristics--well documented, accurate, and credible--which detracts from its reliability. For example, the estimate's accuracy, and therefore the confidence level assigned to the estimate, was lessened by the summary schedule used for the joint cost and schedule risk analysis because it did not provide enough detail to determine how risks were applied to critical project activities. The estimate's credibility was also lessened because officials did not perform a sensitivity analysis that would have identified key drivers of costs, such as workforce size. Program officials believe that it would have been difficult to fully address all best practice characteristics. GAO believes there is time to improve the estimate and enhance the prospects for delivering the project according to plan.

Project officials report that the JWST schedule has 14 months of reserve, which meets Goddard guidance for schedule reserve; however, only 7 of the 14 months are likely to be available for the last three of JWST's five complex integration and test efforts. GAO's prior work shows that the integration and test phases are where problems are commonly found and schedules tend to slip. Given that JWST has a challenging integration and test schedule, this could particularly be likely. The project has made some significant progress in the past year, notably successfully completing development of the 18 primary mirror segments--considered JWST's top technical risk. Nevertheless, ongoing challenges are indicative of the kinds of issues that can require significant effort to address. For example, instrument challenges have delayed the first integration and test effort. In addition, key long-term risks on subsystems with a significant amount of work remaining will not be retired until 2016. Currently, NASA's plan for project oversight calls for one independent mission-level system integration review about 13 months before launch. While this is consistent with what NASA requires for its projects, this approach may not be sufficient for a project as complex as JWST.

JWST has taken several steps to improve communications and oversight of the project and its contractors--such as taking over responsibility for mission systems engineering from the prime contractor; instituting meetings that include various levels of NASA, contractor, and subcontractor management; and implementing a new risk management system to allow for better tracking of risks. The enhancements to the oversight of the project are steps in the right direction, but it will take time to assess their effectiveness and ensure that the efforts are sustained by the project in the future. Reductions in travel budgets, however, could require the project to adjust the oversight approach that was adopted as a result of the replan. Additional reductions in travel budgets are anticipated in future years, but officials do not have a plan to address such reductions and their potential impact on continuing the current oversight approach.

Why GAO Did This Study

JWST is one of NASA's most expensive and technologically advanced science projects, intended to advance understanding of the origin of the universe. In 2011, JWST was rebaselined with a life cycle cost estimate of $8.8 billion and a launch readiness date in October 2018--almost nine times the cost and more than a decade later than originally projected in 1999. Concern about the magnitude of JWST's cost increase and schedule delay and their effects on NASA's progress on other high-priority missions led conferees for the Consolidated and Further Continuing Appropriations Act, 2012, to direct GAO to report on the project. Specifically, GAO assessed (1) the extent to which NASA's revised cost and schedule estimates are reliable based on best practices, (2) the major risks and technological challenges JWST faces, and (3) the extent to which NASA has improved oversight of JWST. To do this, GAO compared NASA's revised cost and schedule estimates with best practice criteria, reviewed relevant contractor and NASA documents, and interviewed project and contractor officials.

What GAO Recommends

GAO recommends NASA take six actions including, among others, to take steps to improve its cost estimate; to conduct an additional, earlier independent review of test and integration activities; and to develop a long-term oversight plan that anticipates planned travel budget reductions.

For more information, contact Cristina Chaplain at (202) 512-4841 or chaplainc@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: The agency has taken steps toward implementation of this recommendation by performing analysis of earned value management data for most contracted and internally performed work, but not all. Within the past year, the JWST project has begun collecting earned value management data for the cryocooler development effort, a component that is experiencing technical issues, and is conducting internal analysis of this data. The agency has plans to develop an analysis tool for work being performed by a non-profit organization in development of the JWST ground systems. However, we have received no indication from project officials that they will use this information to update the cost estimate. This recommendation will reamin open until we determine how the project uses the information it is collecting and whether officials will update the estimate.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to improve cost estimate documentation and continually update it to reflect earned value management actual costs and record any reasons for variances.

    Agency Affected: National Aeronautics and Space Administration

  2. Status: Open

    Comments: In commenting on this recommendation, the agency partially concurred with this recommendation, citing it met the intent, but the agency has not taken actions to implement this recommendation. For example, NASA cites that it took a number of steps to determine how staffing variations affect its cost estimate and to ensure the project has sufficient cost reserves to address changes in workforce requirements. The analysis performed, however, was a part of the cost/schedule risk assessment that was based on information at too high a level of detail to sufficiently account for risk. In addition, the JWST project has used a significant portion of its cost reserves to address technical issues, leaving very little available to address changes in workforce requirements.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to conduct a sensitivity analysis on the number of staff working on the program to determine how staff variations affect the cost estimate.

    Agency Affected: National Aeronautics and Space Administration

  3. Status: Open

    Comments: In commenting on this recommendation, the agency concurred but has not taken actions to implement this recommendation. NASA stated that it performs monthly integrated programmatic and cost/schedule risk analysis. For example, the JWST project identifies risks, assesses encumbrances to its cost reserves, tracks actual costs against planned costs, and performs earned value management and schedule analyses. However, the project does not pull this information together in an integrated assessment to gain a picture of overall progress, which would be in line with cost and schedule risk analysis best practices.

    Recommendation: To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to perform an updated integrated cost/schedule risk analysis, or joint cost and schedule confidence level analysis, using a schedule that meets best practices and includes enough detail so that risks can be appropriately mapped to activities and costs; historical, analogous data should be used to support the risk analysis.

    Agency Affected: National Aeronautics and Space Administration

  4. Status: Closed - Implemented

    Comments: The JWST project has now scheduled an independent systems integration review for Optical Telescope and Integrated Science Instrument Module (OTIS) in May 2016, prior to the beginning of the OTIS integration and test effort that same month. In addition, the JWST project has scheduled the spacecraft element (SCE) readiness review in March 2016, prior to the beginning of the spacecraft integration and test effort currently scheduled to begin in April 2016.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to conduct a separate independent review prior to the beginning of the Optical Telescope Element and Integrated Science Instrument Module and spacecraft integration and test efforts to allow the project's independent standing review board the opportunity to evaluate the readiness of the project to move forward, given the lack of schedule flexibility once these efforts are under way.

    Agency Affected: National Aeronautics and Space Administration

  5. Status: Open

    Comments: In commenting on this recommendation, the agency partially concurred but has not taken actions to implement this recommendation. The agency cites that the Standing Review Board's outbrief of the results of the System Integration Review to the NASA Associate Administrator prior to the start of observatory integration and test meets the intent of this recommendation. The Standing Review Board's outbrief to the NASA Associate Administrator is a courtesy and no action is required at that time. NASA policy, however, clearly states that the key decision point and approval by the NASA Associate Administrator should be held prior to integration and test activities commence. The JWST project has this management review and approval 2 months after the start of observatory integration and test.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to schedule the management review and approval to proceed to integration and test (key decision point D or KDP-D) prior to the start of observatory integration and test effort.

    Agency Affected: National Aeronautics and Space Administration

  6. Status: Closed - Not Implemented

    Comments: In commenting on this recommendation, the agency concurred but has not taken actions to implement this recommendation. The JWST Program Manager said that the project has not incurred any reductions to its travel budget in fiscal years 2013 and 2014, and will under run its travel budget for the 2014 due to efficiencies put in place for its oversight processes and the conclusion of two major contracting efforts. In addition, the project has obtained commitments from the agency that future funding will be in line with baseline projections.

    Recommendation: To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to devise an effective, long-term plan for project office oversight of its contractors that takes into consideration the anticipated travel budget reductions.

    Agency Affected: National Aeronautics and Space Administration

 

Explore the full database of GAO's Open Recommendations »

Dec 15, 2014

Dec 10, 2014

Dec 9, 2014

Jul 23, 2014

Jul 16, 2014

Jun 20, 2014

May 15, 2014

May 8, 2014

Apr 15, 2014

Looking for more? Browse all our products here