Project and Program Management: DOE Needs to Revise Requirements and Guidance for Cost Estimating and Related Reviews
Highlights
What GAO Found
The Department of Energy's (DOE) and its National Nuclear Security Administration's (NNSA) cost estimating requirements and guidance for projects and programs do not fully reflect best practices for developing cost estimates. In regard to cost estimating requirements for projects, DOE's 2010 project management order requires 1 of the 12 cost estimating best practices—conducting an independent cost estimate—for larger projects at certain stages of development. In contrast, DOE's 2011 cost estimating guide describes recommended approaches for using 10 of 12 best practices and partially contains information about the other 2. Furthermore, because DOE's cost estimating guide was issued in 2011—after DOE's 2010 project order was issued—it is not referenced in the order. As a result, users of the order may not be aware of the guide's availability and may not benefit from its usefulness. In addition, although NNSA programs are required to follow DOE's budget formulation order and NNSA's budget process, both of which require the development of cost estimates, neither the order nor the process requires the use of best practices in developing the estimates. In February 2014, for example, GAO found that NNSA's lifecycle cost estimate for the Plutonium Disposition Program did not follow all key steps for developing high-quality cost estimates, in part because the agency did not have a requirement to develop a life-cycle cost estimate. In the absence of a requirement for using best practices, it is unlikely that DOE, NNSA, and their contractors will consistently develop reliable cost estimates.
NNSA's project and program reviews issued during fiscal year 2009 through March 2014 identified cost estimating weaknesses that can be attributed to not following best practices. DOE and NNSA require independent project reviews, including reviews of cost estimates at certain stages of development and at the discretion of project managers. Of the 50 reviews GAO analyzed, 39 identified a total of 113 cost estimating weaknesses. GAO determined that 71 of the 113 weaknesses—or about 63 percent—can be attributed to not following four best practices: (1) determining the estimating structure, (2) identifying ground rules and assumptions, (3) conducting risk and uncertainty analysis, and (4) documenting the estimate. Neither DOE nor NNSA, however, requires reviews of program cost estimates. Of the three program reviews conducted during fiscal years 2009 to 2013, two were of the B61 Life Extension Program, which is to extend the operational life of this nuclear weapon. Both reviews identified weaknesses in the cost estimates that can be attributed to not following three best practices: (1) determining the estimating structure, (2) defining program characteristics, and (3) obtaining data. In addition, a February 2014 GAO report on NNSA's program to dispose of weapons-grade plutonium found that NNSA did not follow several cost estimating best practices, such as conducting an independent cost estimate and as a result, the program cost estimate was not reliable. While the program reviews and GAO's February 2014 report indicate weaknesses in a few program cost estimates, the extent of program cost estimate weaknesses is largely unknown because neither DOE nor NNSA requires reviews of program cost estimates. Without a requirement for conducting independent program reviews, NNSA does not have the internal control necessary for assessing program performance over time.
Why GAO Did This Study
NNSA is responsible for the nation's nuclear security programs. These programs often include the design and construction of large projects to meet program needs. NNSA has a history of struggling to complete these and other projects and programs within cost estimates.
Senate Report 112-73 mandated that GAO report on NNSA's cost estimating practices. This report examines: (1) the extent to which DOE and NNSA cost estimating requirements and guidance for projects and programs reflect best practices and (2) the extent to which recent NNSA project and program reviews identified cost estimating weaknesses, and the extent to which the weaknesses can be attributed to not following best practices. GAO reviewed DOE and NNSA cost estimating practices and compared them to best practices, NNSA project and program reviews, and two programs selected among the largest and most complex NNSA programs. GAO also interviewed DOE and NNSA officials about requirements and guidance for cost estimates.
Recommendations
GAO recommends that DOE, among other things, revise DOE requirements and guidance that apply to projects and programs to ensure: (1) the guidance is referenced in the project management order, (2) DOE and NNSA and its contractors develop cost estimates in accordance with the 12 best practices, (3) its cost estimating guide fully reflects best practices, and (4) independent reviews of programs are conducted periodically. DOE agreed with the report and its recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Energy |
Priority Rec.
To enhance NNSA's ability to develop reliable cost estimates for its projects and for its programs that have project-like characteristics, the Secretary of Energy should revise DOE's project management order to require that DOE, NNSA, and its contractors develop cost estimates in accordance with the 12 cost estimating best practices. |
In June 2015, based in part upon our work, the Secretary of Energy issued a memo to enhance and clarify departmental policy related to project management. The memo outlined changes that recent GAO reports had noted as areas for improvement. Specifically, the memo required that the DOE project management order, the cost estimating guide, and the Department of Energy Acquisition Regulations be revised consistent with the cost estimating best practices. The memo further specified that these and other provisions of the memo were to be made effective immediately and implemented as required project management procedures. This action satisfies this recommendation.
|
| Department of Energy | To enhance NNSA's ability to develop reliable cost estimates for its projects and for its programs that have project-like characteristics, the Secretary of Energy should revise DOE's cost estimating guide so that it fully reflects the 12 cost estimating best practices. |
In June 2015, based in part upon our work, the Secretary of Energy issued a memo to enhance and clarify departmental policy related to project management. The memo outlined changes that recent GAO reports had noted as areas for improvement. Specifically, the memo required that the DOE project management order, the cost estimating guide, and the Department of Energy Acquisition Regulations be revised consistent with the cost estimating best practices. The memo further specified that these and other provisions of the memo were to be made effective immediately and implemented as required project management procedures. This action satisfies this recommendation.
|
| Department of Energy | To enhance NNSA's ability to develop reliable cost estimates for its projects and for its programs that have project-like characteristics, the Secretary of Energy should revise DOE's project management order to include references to the DOE cost estimating guide, where applicable. |
DOE issued a revised project management order (Order 413.3B, change 2) on 5/12/16 and the revisions included references to the DOE cost estimating guide (DOE G 413.3-21).
|
| Department of Energy |
Priority Rec.
To enhance NNSA's ability to develop reliable cost estimates for its projects and for its programs that have project-like characteristics, the Secretary of Energy should revise DOE directives that apply to programs to require that DOE and NNSA and its contractors develop cost estimates in accordance with the 12 cost estimating best practices, including developing life-cycle cost estimates for programs. |
In February 2019, NNSA issued two directives that establish policies for conducting program management activities and independent cost estimates and reviews within NNSA. One directive (NNSA Policy 413.3) requires NNSA to conduct independent cost estimates, consistent with GAO's best practices for cost estimating, for acquisition programs at program milestones. It also requires NNSA to include life-cycle cost estimates as part of independent cost reviews of acquisition programs. The other directive (NNSA Policy 413.2) requires all NNSA programs to establish and document requirements for cost management, including developing cost estimates that cover the life cycle of programs where appropriate. Further, in establishing requirements for cost management, NNSA programs are to consider the use of GAO's cost estimating best practices. Because all of the programs we reviewed in our report were NNSA programs, and because these directives apply to NNSA programs, we believe that the agency's actions satisfy the intent of our recommendation to improve NNSA's development of programmatic cost estimates.
|
| Department of Energy |
Priority Rec.
To enhance NNSA's ability to develop reliable cost estimates for its projects and for its programs that have project-like characteristics, the Secretary of Energy should revise DOE requirements and guidance that apply to programs to ensure that program reviews are conducted periodically, including reviews of the life-cycle cost estimates for programs. |
In February 2019, NNSA issued two directives that establish policies for conducting program management activities and independent cost estimates and reviews within NNSA. One directive (NNSA Policy 413.2) requires all NNSA programs to establish and document requirements for periodic program reviews with the Administrator and with the relevant Deputy or Associate Administrator. The other directive (NNSA Policy 413.3) requires NNSA to conduct independent cost reviews of acquisition programs at program milestones, which would include the development of life-cycle cost estimates. Because all of the programs we reviewed in our report were NNSA programs, and because these directives apply to NNSA programs, we believe that the agency's actions satisfy the intent of our recommendation to improve NNSA's programmatic reviews of cost estimates.
|