National Science Foundation: Revised Policies on Developing Costs and Schedules Could Improve Estimates for Large Facilities
Fast Facts
One of the ways the National Science Foundation supports research is by funding construction of major research equipment and facilities, such as telescopes and ships with construction costs of $70 million or more.
We looked at whether such projects have stayed on track since the agency's 2008 policy to manage construction cost overruns. We found that 5 of 7 projects had cost or schedule increases.
We also looked at NSF's procedures for construction projects and found that they met many but not all best practices for estimating costs and schedules.
We recommended that NSF better incorporate estimating best practices into its policies.
NSF's Daniel K. Inouye Solar Telescope on Maui, Hawaii is more than 80% complete. It has experienced a 16% cost increase and a 31% schedule delay.
This is a photo of a telescope building on top of a mountain and above clouds.
Highlights
What GAO Found
The National Science Foundation's (NSF) procedures for overseeing large facilities construction projects met many best practices for cost estimating but not those for developing project schedules. Specifically, NSF's procedures fully or substantially met 7 of 12 best practices in GAO's cost estimating guide and partially or minimally met others (such as conducting a sensitivity analysis to understand which variables most affect the cost estimate). In addition, they minimally met or did not meet 6 of 10 best practices in GAO's schedule development guide (such as establishing the durations of all activities). Further, while NSF reviews recipients' construction cost and schedule estimates for large facilities, the agency's policies did not incorporate procedures on how NSF officials are to ensure that those estimates meet best practices. An agency's procedures support the creation of reliable cost and schedule estimates when they fully or substantially meet the best practices in GAO's cost and schedule guides. NSF officials said that the agency's approach was to reference rather than repeat best practices in GAO's cost guide, except where agency-specific clarifications were needed. Nevertheless, without policies on how to apply all relevant best practices specifically to NSF's large facilities projects, recipients may develop cost or schedule estimates that are not reliable.
Of the seven projects NSF had funded that were covered by its policy to manage cost overruns, five had experienced cost or schedule increases since starting construction. In particular, two projects—the National Ecological Observatory Network and Daniel K. Inouye Solar Telescope—had both cost and schedule increases that were partly due to permitting challenges that could not be estimated at the time of the projects' proposals, according to NSF officials. Both of these projects also reduced their scopes. NSF required the reductions to the observatory network's scope under the agency's policy for managing cost overruns, whereas the reduction to the telescope project was a separate action. Three other projects had only schedule increases, with no increase in costs, and the remaining two projects had experienced neither cost nor schedule increases as of December 2017.
Cost and Schedule Performance of Large Facilities Projects Covered by NSF's Policy for Managing Cost Overruns, as of December 2017
Project name |
Percentage complete |
Cost change |
Schedule change |
Scope reductions |
Alaska Region Research Vessel |
100 |
▼ |
▲ |
- |
Ocean Observatories Initiative |
100 |
▼ |
▲ |
- |
Advanced Laser Interferometer Gravitational Wave Observatory |
99 |
- |
▲ |
- |
National Ecological Observatory Network |
94 |
▲ |
▲ |
✔ |
Daniel K. Inouye Solar Telescope |
81 |
▲ |
▲ |
✔ |
Large Synoptic Survey Telescope |
50 |
- |
- |
- |
Regional Class Research Vessels |
4 |
- |
- |
- |
Legend: - = no cost or schedule increase or scope reductions since starting construction; ▼= cost decreased; ▲ = cost or schedule increased; ✔ = scope reduced.
Source: GAO analysis of National Science Foundation (NSF) documents. | GAO-18-370
Why GAO Did This Study
NSF uses cooperative agreements and contracts to fund construction of science and engineering research infrastructure, such as telescopes. In 2008, NSF established a policy to manage cost overruns and strengthen oversight of these large facilities projects, which typically have construction costs greater than $70 million. Among other things, the policy generally requires a project's scope to be reduced before its NSF-authorized cost may be increased.
Senate Report 114-239 and House Report 114-605 included provisions for GAO to review projects funded within NSF's Major Research Equipment and Facilities Construction account used to fund construction of large facilities projects. GAO (1) examined the extent to which NSF's procedures for estimating construction costs and schedules of large facilities met best practices and (2) described the construction cost and schedule performance of NSF's large facilities projects since implementation of its policy to manage cost overruns. GAO compared NSF's procedures documented in its policies with best practices in GAO's cost and schedule guides, analyzed documents for the seven projects covered by NSF's policy to manage cost overruns, and interviewed NSF officials.
Recommendations
GAO recommends that NSF revise its policies for estimating and reviewing the costs and schedules of large facilities projects to better incorporate best practices in GAO's cost and schedule guides. NSF agreed with GAO's recommendations.
Recommendations for Executive Action
Agency Affected | Recommendation | Status |
---|---|---|
National Science Foundation | The Director of NSF should revise the agency's policies for estimating the costs of large facilities projects, and for reviewing those costs, to better incorporate the best practices in GAO's cost guide. (Recommendation 1) |
NSF addressed this recommendation by updating its Major Facilities Guide (previously named the Large Facilities Manual) in September 2019 to fully incorporate the five cost estimating best practices in GAO's cost guide that we previously found were minimally or partially met. For example, whereas NSF's procedures previously required a sensitivity analysis but did not describe how one is to be conducted, its updates to the Major Facilities Guide now also describe how sensitivity analysis should be applied to major facilities' cost estimates.
|
National Science Foundation | The Director of NSF should revise the agency's policies for developing schedules for large facilities projects, and for reviewing those schedules, to better incorporate the best practices in GAO's schedule guide. (Recommendation 2) |
In December 2021, NSF revised the Research Infrastructure Guide, the agency's main guidance document for oversight, to include a new section on developing schedules that incorporates best practices in GAO's schedule guide. In addition, NSF updated internal guidance requiring the agency to analyze proposed schedules. These actions address a recommendation from our June 2018 report where we found that NSF substantially met one of the 10 best practices for developing project schedules, partially or minimally met six best practices, and did not meet three best practices.
|