James Webb Space Telescope: Actions Needed to Improve Cost Estimate and Oversight of Test and Integration
Highlights
What GAO Found
The National Aeronautics and Space Administration (NASA) has provided significantly more time and money to the James Webb Space Telescope (JWST) than previously planned and expressed high confidence in the project's new baselines. Its current cost estimate reflects some features of best practices for developing reliable and credible estimates. For example, the estimate substantially meets one of four cost characteristics--comprehensive--that GAO looks for in a reliable cost estimate, in part because all life cycle costs were included. The estimate, however, only partially met the other three characteristics--well documented, accurate, and credible--which detracts from its reliability. For example, the estimate's accuracy, and therefore the confidence level assigned to the estimate, was lessened by the summary schedule used for the joint cost and schedule risk analysis because it did not provide enough detail to determine how risks were applied to critical project activities. The estimate's credibility was also lessened because officials did not perform a sensitivity analysis that would have identified key drivers of costs, such as workforce size. Program officials believe that it would have been difficult to fully address all best practice characteristics. GAO believes there is time to improve the estimate and enhance the prospects for delivering the project according to plan.
Project officials report that the JWST schedule has 14 months of reserve, which meets Goddard guidance for schedule reserve; however, only 7 of the 14 months are likely to be available for the last three of JWST's five complex integration and test efforts. GAO's prior work shows that the integration and test phases are where problems are commonly found and schedules tend to slip. Given that JWST has a challenging integration and test schedule, this could particularly be likely. The project has made some significant progress in the past year, notably successfully completing development of the 18 primary mirror segments--considered JWST's top technical risk. Nevertheless, ongoing challenges are indicative of the kinds of issues that can require significant effort to address. For example, instrument challenges have delayed the first integration and test effort. In addition, key long-term risks on subsystems with a significant amount of work remaining will not be retired until 2016. Currently, NASA's plan for project oversight calls for one independent mission-level system integration review about 13 months before launch. While this is consistent with what NASA requires for its projects, this approach may not be sufficient for a project as complex as JWST.
JWST has taken several steps to improve communications and oversight of the project and its contractors--such as taking over responsibility for mission systems engineering from the prime contractor; instituting meetings that include various levels of NASA, contractor, and subcontractor management; and implementing a new risk management system to allow for better tracking of risks. The enhancements to the oversight of the project are steps in the right direction, but it will take time to assess their effectiveness and ensure that the efforts are sustained by the project in the future. Reductions in travel budgets, however, could require the project to adjust the oversight approach that was adopted as a result of the replan. Additional reductions in travel budgets are anticipated in future years, but officials do not have a plan to address such reductions and their potential impact on continuing the current oversight approach.
Why GAO Did This Study
JWST is one of NASA's most expensive and technologically advanced science projects, intended to advance understanding of the origin of the universe. In 2011, JWST was rebaselined with a life cycle cost estimate of $8.8 billion and a launch readiness date in October 2018--almost nine times the cost and more than a decade later than originally projected in 1999. Concern about the magnitude of JWST's cost increase and schedule delay and their effects on NASA's progress on other high-priority missions led conferees for the Consolidated and Further Continuing Appropriations Act, 2012, to direct GAO to report on the project. Specifically, GAO assessed (1) the extent to which NASA's revised cost and schedule estimates are reliable based on best practices, (2) the major risks and technological challenges JWST faces, and (3) the extent to which NASA has improved oversight of JWST. To do this, GAO compared NASA's revised cost and schedule estimates with best practice criteria, reviewed relevant contractor and NASA documents, and interviewed project and contractor officials.
Recommendations
GAO recommends NASA take six actions including, among others, to take steps to improve its cost estimate; to conduct an additional, earlier independent review of test and integration activities; and to develop a long-term oversight plan that anticipates planned travel budget reductions.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| National Aeronautics and Space Administration |
Priority Rec.
To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to improve cost estimate documentation and continually update it to reflect earned value management actual costs and record any reasons for variances. |
The JWST program has collected earned value management data for its prime contract for several years since a major program replan in 2011, developed an earned value management-like tool for monitoring internal work, and developed supplemental analysis to improve the oversight of its contractors. These actions, however, do not meet the intent of the recommendation because the program has not implemented a mechanism to update the cost estimate by incorporating actual contractor or in-house costs. Over three years have passed since we first made the recommendation and we believe there is no longer sufficient time for the JWST program to take action. The program is scheduled for launch in October 2018, but much of the integration and test activities are scheduled to be completed about a year from now. The program has less than 10 percent of its life cycle cost remaining and the prime contractor has less than 6 percent of its costs remaining; therefore, the program no longer has time to benefit from implementing this recommendation. We still believe NASA should ensure life-cycle cost estimates conform to best practices, including continually updating cost estimates for all its programs, but this program is too close to launch for NASA to invest resources in updating the cost estimate.
|
| National Aeronautics and Space Administration | To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to conduct a sensitivity analysis on the number of staff working on the program to determine how staff variations affect the cost estimate. |
The JWST project does not expect to perform the workforce sensitivity analysis as recommended by GAO. The JWST project has used a significant portion of its cost reserves to address technical issues, with some of the cost reserves being used to account for an overage in workforce requirements. The prime contractor has carried a significantly higher workforce, much at higher rates of pay such as engineers, than planned. The JWST program manager indicated that the monthly workforce peak for this contractor is close, but when that will happen is unknown at this time.
|
| National Aeronautics and Space Administration | To ensure that the JWST life-cycle cost estimate conforms to best practices, and to to provide high-fidelity cost information for monitoring project progress, the NASA Administrator should direct JWST officials to perform an updated integrated cost/schedule risk analysis, or joint cost and schedule confidence level analysis, using a schedule that meets best practices and includes enough detail so that risks can be appropriately mapped to activities and costs; historical, analogous data should be used to support the risk analysis. |
The agency concurred with this recommendation, but the the project has not pulled this information together in an integrated assessment to gain a picture of overall progress, which would be in line with cost and schedule risk analysis best practices. Since so much time has elapsed, the JWST project does enough time left in their schedule to perform an updated joint cost/schedule confidence level analysis in time to be of value to the project.
|
| National Aeronautics and Space Administration | To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to conduct a separate independent review prior to the beginning of the Optical Telescope Element and Integrated Science Instrument Module and spacecraft integration and test efforts to allow the project's independent standing review board the opportunity to evaluate the readiness of the project to move forward, given the lack of schedule flexibility once these efforts are under way. |
The JWST project has now scheduled an independent systems integration review for Optical Telescope and Integrated Science Instrument Module (OTIS) in May 2016, prior to the beginning of the OTIS integration and test effort that same month. In addition, the JWST project has scheduled the spacecraft element (SCE) readiness review in March 2016, prior to the beginning of the spacecraft integration and test effort currently scheduled to begin in April 2016.
|
| National Aeronautics and Space Administration | To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to schedule the management review and approval to proceed to integration and test (key decision point D or KDP-D) prior to the start of observatory integration and test effort. |
In April 2017, NASA changed the James Webb Space Telescope (JWST) project schedule and will now hold management review (key decision point D or KDP-D) in November 2017 before the start of JWST's integration and test effort in December 2017. KDP-D is when NASA's senior agency decision authority would approve the project to proceed into the system integration and test phase. The change will place the project in compliance NASA policy which states that the key decision point and approval by the NASA Associate Administrator should be held prior to integration and test activities commence.
|
| National Aeronautics and Space Administration | To ensure that technical risks and challenges are being effectively managed and that sufficient oversight is in place and can be sustained, the NASA Administrator should direct JWST officials to devise an effective, long-term plan for project office oversight of its contractors that takes into consideration the anticipated travel budget reductions. |
In commenting on this recommendation, the agency concurred but has not taken actions to implement this recommendation. The JWST Program Manager said that the project has not incurred any reductions to its travel budget in fiscal years 2013 and 2014, and will under run its travel budget for the 2014 due to efficiencies put in place for its oversight processes and the conclusion of two major contracting efforts. In addition, the project has obtained commitments from the agency that future funding will be in line with baseline projections.
|