This is the accessible text file for GAO report number GAO-07-336 entitled 'Department of Energy: Major Construction Projects Need a Consistent Approach for Assessing Technology Readiness to Help Avoid Cost Increases and Delays' which was released on March 28, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Subcommittee on Energy and Water Development, and Related Agencies, Committee on Appropriations, House of Representatives: United States Government Accountability Office: GAO: March 2007: Department of Energy: Major Construction Projects Need a Consistent Approach for Assessing Technology Readiness to Help Avoid Cost Increases and Delays: GAO-07-336: GAO Highlights: Highlights of GAO-07-336, a report to the Subcommittee on Energy and Water Development, and Related Agencies, Committee on Appropriations, House of Representatives Why GAO Did This Study: The Department of Energy (DOE) spends billions of dollars on major construction projects that help maintain the nuclear weapons stockpile, conduct research and development, and process nuclear waste so that it can be disposed of. Because of DOE’s long-standing project management problems, GAO determined the extent to which (1) DOE’s major construction projects are having cost increases and schedule delays and the major factors contributing to these problems and (2) DOE ensures that project designs are sufficiently complete before construction begins to help avoid cost increases and delays. We examined 12 DOE major projects with total costs of about $27 billion, spoke with federal and contractor officials, and reviewed project management documents. What GAO Found: Of the 12 DOE major projects GAO reviewed, 9 exceeded their original cost or schedule estimates, principally because of ineffective DOE project oversight and poor contractor management. Specifically, 8 of the 12 projects experienced cost increases ranging from $79.0 million to $7.9 billion, and 9 of the 12 projects were behind schedule by 9 months to more than 11 years. Project oversight problems included, among other things, inadequate systems for measuring contractor performance, approval of construction activities before final designs were sufficiently complete, ineffective project reviews, and insufficient DOE staffing. Furthermore, contractors poorly managed the development and integration of the technology used in the projects by, among other things, not accurately anticipating the cost and time that would be required to carry out the highly complex tasks involved. Even though DOE requires final project designs to be sufficiently complete before beginning construction, it has not systematically ensured that the critical technologies reflected in these designs have been demonstrated to work as intended (technology readiness) before committing to construction expenses. Specifically, only one of the five DOE project directors with projects that have recently begun or are nearing construction had systematically assessed technology readiness. The other four directors also told us that they have or will have completed prior to construction, 85 to 100 percent of their projects’ final design, but they had not systematically assessed technology readiness. Proceeding into construction without also demonstrating a technology’s readiness can lead to cost increases and delays. For example, one technology to be used in DOE’s Waste Treatment and Immobilization Plant was not sufficiently demonstrated—that is, shown to be technologically ready for its intended application—before construction began. Consequently, the technology did not perform as expected, which resulted in about $225 million in redesign costs and schedule delays of more than 1 year. To help avoid these problems, the National Aeronautics and Space Administration (NASA) pioneered and the Department of Defense (DOD) has adopted for its projects a method for measuring and communicating technology readiness levels (TRL). Using a scale from one (basic principles observed) through nine (total system used successfully in project operations), TRLs show the extent to which technologies have been demonstrated to work as intended in the project. DOE project directors agreed that such an approach would help make technology assessments more transparent and improve stakeholder communication prior to making critical project decisions, such as authorizing construction. Figure: Technology Readiness Levels: [See PDF for Image] Source: GAO analysis of DOD data. [End of figure] What GAO Recommends: GAO recommends that DOE develop a consistent approach for measuring the readiness of critical project technologies. DOE supports GAO’s recommendations but suggested revisions to allow it to first conduct a pilot application on selected projects to better understand the process and evaluate its potential use. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-336]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Gene Aloise at (202) 512- 3841 or aloisee@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: Most Major Projects Have Exceeded Original Costs and Are Years Late, Principally Because of Ineffective DOE Project Oversight and Contractor Management: DOE Does Not Consistently Measure Technology Readiness to Ensure That Critical Technologies Will Work as Intended before Construction Begins: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Information on the 12 Department of Energy Major Projects Reviewed: Appendix III: Independent Studies Reviewed: Appendix IV: Survey Results for Primary Factors Affecting Cost and Schedule on Nine Projects with Cost or Schedule Changes: Appendix V: Definitions of Technology Readiness Levels: Appendix VI: Comparison of DOD's Product Development Process with DOE's Project Management Process: Appendix VII: Comments from the Department of Energy: Appendix VIII: GAO Contact and Staff Acknowledgments: Tables: Table 1: Changes in Estimated Total Project Cost for DOE Major Construction Projects: Table 2: Changes in Estimated Project Schedules for DOE Major Construction Projects: Table 3: Reasons for Cost Increases and Schedule Delays: Abbreviations: DOD: Department of Defense: DOE: Department of Energy: EM: Office of Environmental Management: ITP: In-Tank Precipitation: NASA: National Aeronautics and Space Administration: NNSA: National Nuclear Security Administration: PDRI: Product Definition Rating Index: TPC: total project cost: TRL: technology readiness level: United States Government Accountability Office: Washington, DC 20548: March 27, 2007: The Honorable Peter J. Visclosky: Chairman: The Honorable David L. Hobson: Ranking Member: Subcommittee on Energy and Water Development, and Related Agencies: Committee on Appropriations: House of Representatives: The Department of Energy (DOE) spends billions of dollars on major construction projects that, among other things, are used to help maintain the nuclear weapons stockpile, conduct research and development in the areas of high-energy physics and nuclear physics, and process nuclear waste into forms suitable for longer-term storage or permanent disposal. DOE oversees the construction of facilities primarily at government-owned, contractor-operated sites throughout the nation. In July 2006, DOE revised its dollar threshold defining a construction project as "major"; it is now at $750 million, up from $400 million when we began our review in December 2005. The following 12 projects included in our review had estimated project costs exceeding the original threshold.[Footnote 1] The total cost of these projects is currently estimated at about $27 billion.[Footnote 2] These 12 projects and their locations are as follows: * Chemistry and Metallurgy Research Facility Replacement--Los Alamos National Laboratory, Los Alamos, New Mexico. * Depleted Uranium Hexafluoride 6 Conversion Facility--Portsmouth, Ohio, and Paducah, Kentucky. * Highly Enriched Uranium Materials Facility--Y-12 National Security Complex, Oak Ridge, Tennessee. * Linac Coherent Light Source--Stanford Linear Accelerator Center, Menlo Park, California. * Microsystems and Engineering Sciences Applications--Sandia National Laboratories, Albuquerque, New Mexico. * Mixed Oxide Fuel Fabrication Facility--Savannah River Site, Aiken, South Carolina. * National Ignition Facility--Lawrence Livermore National Laboratory, Livermore, California. * Pit Disassembly and Conversion Facility--Savannah River Site, Aiken, South Carolina. * Salt Waste Processing Facility--Savannah River Site, Aiken, South Carolina. * Spallation Neutron Source--Oak Ridge National Laboratory, Oak Ridge, Tennessee. * Tritium Extraction Facility--Savannah River Site, Aiken, South Carolina. * Waste Treatment and Immobilization Plant--Hanford Site, Richland, Washington. These major projects require the construction of large building complexes and the development of innovative cleanup and other technologies. Many of these technologies are developed for the project or are applied in a new way. DOE project directors are responsible for managing these major projects and overseeing the contractors that design and construct the facilities. In doing so, project directors follow specific departmental directives, policies, and guidance for project management. Among these are DOE Order 413.3A and Manual 413.3- 1, which establish protocols for planning and executing a project. The protocols require DOE projects to go through a series of five critical decisions as they enter each new phase of work. Two of the decisions made before construction begins are key: (1) formally approving the project's definitive cost and schedule estimates as accurate and complete--an approval that is to be based on a review of the project's completed preliminary design, and (2) reaching agreement that the project's final design is sufficiently complete and that resources can be committed toward procurement and construction. To oversee projects and approve these critical decisions, DOE conducts its own reviews, often with the help of independent technical experts. In addition, projects are regularly subject to reviews by DOE's Office of Engineering and Construction Management and it's Office of Inspector General, the Department of Defense's (DOD) U.S. Army Corps of Engineers, the Defense Nuclear Facilities Safety Board, and the National Research Council, among others. We and others have reported over the past decade that project management weaknesses have impaired these projects. For example, for the Waste Treatment and Immobilization Plant, we reported that DOE's use of a "fast-track, design-build" approach to construction--in which design and construction activities overlap--has been problematic for highly complex, first-of-a-kind facilities. We found that the designs for these facilities were not sufficiently complete for construction to begin, which has resulted in significant cost increases and schedule delays.[Footnote 3] In this context, we determined the extent to which (1) DOE's active major construction projects are experiencing cost increases and schedule delays and the key factors contributing to these problems and (2) DOE ensures that project designs are sufficiently complete before construction begins to help avoid cost increases and schedule delays.[Footnote 4] To determine the extent to which DOE projects are experiencing cost increases or schedule delays and factors contributing to these problems, we sent a survey to the 12 DOE directors of major projects and reviewed the project management documents for these projects--6 projects that were above the $750 million threshold, 2 estimated to cost between the previous $400 million threshold and $750 million, and 4 estimated to cost between $300 million and $400 million. (App. II describes these projects.) These 12 projects are managed by DOE's Office of Science, Office of Environmental Management, or National Nuclear Security Administration (NNSA). We conducted site visits and analyzed independent project studies for the 9 projects that experienced cost increases or schedule delays. During the course of our review, we identified a method used by the DOE project director for the Pit Disassembly and Conversion Facility to systematically assess the extent to which a technology is sufficiently developed for its intended purpose. The project director based this method on a system developed by the National Aeronautics and Space Administration (NASA). We had previously reported on the use of a similar assessment system-- technology readiness levels (TRL), which DOD has adopted for its major projects.[Footnote 5] We obtained and reviewed documents regarding these two assessment systems. To determine the extent to which DOE ensures that project designs are sufficiently complete before construction, we reviewed in detail 5 of the 12 projects that were approaching or had recently begun construction. Specifically, we obtained information on the extent to which project designs were, or are expected to be, complete before beginning construction, and the actions DOE has taken to ensure that technologies used in these designs are sufficiently ready to begin construction. Because we and others have previously expressed concern about the data reliability of a key DOE project management tracking database--the Project Assessment and Reporting System--we did not develop conclusions or findings on the basis of information generated through that system.[Footnote 6] Instead, we collected information directly from project site offices. In addition, we spoke with officials from DOE program offices and DOE's Office of Engineering and Construction Management in Washington, D.C. We provided interim briefings to the Subcommittee on the status of our work in May and September, 2006. We performed our work between December 2005 and January 2007, in accordance with generally accepted government auditing standards. Appendix I contains a detailed description of our scope and methodology. Results in Brief: Nine of the 12 DOE major projects we reviewed have exceeded their original cost estimates and/or experienced schedule delays, principally because of ineffective DOE project oversight and poor contractor management, according to independent studies we reviewed and interviews we conducted with DOE and contractor project officials. Specifically, 8 of the 12 projects experienced cost increases ranging from $79.0 million to $7.9 billion, and 9 of the 12 projects are behind schedule by 9 months to more than 11 years. Major factors cited for these cost increases and delays included the following: * Ineffective DOE project oversight. For all 9 projects experiencing cost increases or schedule delays, poor DOE oversight was a key contributing factor. Project oversight problems included inadequate systems for measuring contractor performance, approval of construction activities before final designs were sufficiently complete, ineffective project reviews, and insufficient DOE staffing and project management experience. * Poor contractor management. Eight of the 9 major projects experienced cost increases and/or schedule delays, in part because contractors did not effectively manage the development and integration of the technology used in the projects, including not accurately anticipating the cost and time that would be required to carry out the highly complex tasks involved. For example, the National Ignition Facility has had over $1 billion in cost increases and years of schedule delays owing in part to technology integration problems, according to the DOE project director. Other examples of poor contractor performance included inadequate quality assurance for the Highly Enriched Uranium Materials Facility, which resulted in concrete work that did not meet design specifications. The subsequent suspension of construction activities and rework added to the project's estimated cost and schedule. DOE officials also explained that a now-defunct policy may have contributed to increased costs and delays for several projects we examined. Until 2000, DOE required contractors to prepare cost and schedule estimates early in the project, before preliminary designs were completed. These estimates were used to establish a baseline for measuring contractor performance and tracking any cost increases or schedule delays. However, these estimates often were based on early conceptual designs and, thus, were subject to significant change as more detailed designs were developed. To improve the reliability of these estimates, DOE issued a new order in October 2000 that required the preparation of a cost estimate range at the start of preliminary design, and delayed the requirement for a definitive cost and schedule baseline estimate until after preliminary design was completed. Consequently, DOE officials explained, the new policy should result in improved estimates and a more accurate measure of cost and schedule performance. Even though DOE requires final project designs to be sufficiently complete before beginning construction, it has not systematically ensured that the critical technologies reflected in these designs have been demonstrated to work as intended (technology readiness) before committing to construction expenses. Only one of the five DOE directors with projects that have recently begun, or are nearing construction, had systematically assessed technology readiness. The other four directors also told us that they have or will have completed prior to construction, 85 to 100 percent of their projects' final design, but they had not systematically assessed for technology readiness. Lack of technology readiness can result in cost overruns and schedule delays. For example, technology used in a subsystem intended to prepare radioactive material for processing in DOE's Waste Treatment and Immobilization Plant was not fully developed and did not work as expected after construction had already begun, resulting in redesign costs of about $225 million and over 1 year in schedule delays. To effectively assess technology readiness, NASA pioneered and DOD has adopted a process for measuring and communicating technology readiness for first-of-a-kind technology applications. This process uses a nine- point scale for assessing TRLs. Using this scale, a technology would receive a higher TRL value (e.g., TRL 7) if it has been successfully demonstrated in an operational environment, compared with a technology that has been demonstrated only in a laboratory test (e.g., TRL 4). Several DOE project directors we spoke with agreed that a consistent, systematic method for assessing technology readiness would help standardize terminology, make technology assessments more transparent, and help improve communication among project stakeholders before they make critical project decisions. To improve oversight and decision making for DOE's major construction projects, we are recommending that the Secretary of Energy evaluate and consider adopting a disciplined and consistent approach to assessing TRLs for projects with critical technologies. DOE provided comments to us based on a draft of the report. DOE agreed with our recommendations but suggested revisions that would first allow them to conduct a pilot application on selected projects to better understand the technology readiness assessment process and evaluate its potential use. We revised our recommendations as appropriate. DOE suggested that our report is too narrowly focused on technology assessment, and that we inappropriately calculated cost increases and schedule delays using preliminary estimates that were only intended for internal DOE planning. We believe that our recommendations were justifiably based on our finding that DOE has not systematically ensured that project designs, including critical technologies reflected in these designs, have been demonstrated to work as intended prior to construction. We also believe it was appropriate, when necessary, to measure cost and schedule changes using the initial estimates that were developed at the end of conceptual design, as specified in DOE's project management policy in effect prior to 2000. We note that these estimates were, in some instances, the only initial estimates available and had been used by DOE to inform the Congress of the estimated cost and schedule of the projects while it was seeking initial project funding. We also incorporated technical changes in this report where appropriate on the basis of detailed comments provided by DOE. Background: To meet its diverse missions, DOE pays its contractors billions of dollars each year to implement hundreds of projects, ranging from hazardous waste cleanups at sites in the weapons complex to the construction of scientific facilities. Many of these complex, unique projects are designed to meet defense, energy research, environmental, and fissile materials disposition goals. They often rely on technologies that are unproven in operational conditions. In recent years, DOE's budget has been dominated by the monumental task of environmental restoration and waste management to repair damage caused by the past production of nuclear weapons. DOE has long had a poor track record for developing designs and cost estimates and managing projects. We reported in 1997 that from 1980 to 1996, 31 of DOE's 80 major projects were terminated prior to completion, after expenditures of over $10 billion; 15 of the projects were completed, but most of them were finished behind schedule and with cost overruns; and the remaining 34 ongoing projects also were experiencing schedule slippage or cost overruns.[Footnote 7] In addition, for over a decade, DOE's Office of Inspector General, the National Academy of Sciences, and others have identified problems with DOE's management of major construction projects. Projects were late or never finished; project costs increased by millions and sometimes billions of dollars; and environmental conditions at the sites did not significantly improve. According to the National Research Council,[Footnote 8] DOE's construction and environmental remediation projects take much longer and cost about 50 percent more than comparable projects by other federal agencies or projects in the private sector.[Footnote 9] A 2004 assessment of departmental project management completed by the Civil Engineering Research Foundation recommended, among other things, that DOE develop a core group of highly qualified project directors and require peer reviews for first- of-a-kind and technically complex projects when the projects' preliminary baselines are approved.[Footnote 10] To address project management issues, DOE began a series of reforms in the 1990s that included efforts to strengthen project management practices. To guide these reforms, the department formed the Office of Engineering and Construction Management in 1999. The reforms instituted to date have included planning, organizing, and tracking project activities, costs, and schedules; training to ensure that federal project managers had the required expertise to manage projects; increasing emphasis on independent reviews; and strengthening project reporting and oversight. Most Major Projects Have Exceeded Original Costs and Are Years Late, Principally Because of Ineffective DOE Project Oversight and Contractor Management: The estimated costs of many of the DOE major construction projects we reviewed have significantly exceeded original estimates and schedules have slipped. On the basis of our analysis of independent project studies and interviews with project directors, cost growth and schedule slippage occurred principally because of ineffective DOE project oversight and poor contractor project management. Furthermore, unreliable initial cost and schedule estimates resulting from a now- defunct policy may have been a contributing factor, according to DOE project officials. Although external factors, such as additional security and safety requirements, contributed to cost growth and delays, the management of these requirements was complicated by ineffective and untimely DOE communication and decision making. Eight of the 12 DOE projects we reviewed had increases in estimates of total project cost (TPC) ranging from $79.0 million to $7.9 billion. As table 1 shows, the percentage of cost increase for these 8 projects ranged from 2 percent to over 200 percent. Table 1: Changes in Estimated Total Project Cost for DOE Major Construction Projects: Dollars in millions. Project: Mixed Oxide Fuel Fabrication Facility[C]; Initial total project cost (TPC) estimate[A]: $1,400; Current TPC estimate: $4,699; Percentage increase[B]: 205%. Project: Waste Treatment and Immobilization Plant; Initial total project cost (TPC) estimate[A]: 4,350; Current TPC estimate: 12,263; Percentage increase[B]: 143. Project: Highly Enriched Uranium Materials Facility; Initial total project cost (TPC) estimate[A]: 251; Current TPC estimate: 549; Percentage increase[B]: 102. Project: National Ignition Facility; Initial total project cost (TPC) estimate[A]: 1,199; Current TPC estimate: $2,248; Percentage increase[B]: 59. Project: Salt Waste Processing Facility; Initial total project cost (TPC) estimate[A]: 440; Current TPC estimate: 680[D]; Percentage increase[B]: 50. Project: Pit Disassembly and Conversion Facility[C]; Initial total project cost (TPC) estimate[A]: 1,700; Current TPC estimate: 2,694; Percentage increase[B]: 40. Project: Tritium Extraction Facility; Initial total project cost (TPC) estimate[A]: 384; Current TPC estimate: 506; Percentage increase[B]: 15. Project: Spallation Neutron Source; Initial total project cost (TPC) estimate[A]: 1,333; Current TPC estimate: 1,412; Percentage increase[B]: 2. Project: Depleted Uranium Hexafluoride 6 Conversion Facility; Initial total project cost (TPC) estimate[A]: 346; Current TPC estimate: 346; Percentage increase[B]: 0. Project: Chemistry and Metallurgy Research Facility Replacement; Initial total project cost (TPC) estimate[A]: 837; Current TPC estimate: 837; Percentage increase[B]: 0. Project: Microsystems and Engineering Sciences Applications; Initial total project cost (TPC) estimate[A]: 518; Current TPC estimate: 518; Percentage increase[B]: 0. Project: Linac Coherent Light Source; Initial total project cost (TPC) estimate[A]: 379; Current TPC estimate: 379; Percentage increase[B]: 0. Source: GAO analysis of DOE data. [A] In 2000, DOE changed its requirements for establishing initial cost and schedule estimates. Prior to 2000, these estimates were established at the end of conceptual design. After 2000, DOE required initial estimates to be completed later in the project--at the end of preliminary design. For projects beginning prior to 2000, and for projects beginning after 2000 that had not yet completed preliminary design, we used the TPC estimates prepared after conceptual design. For additional details on our methodology, see appendix I. [B] We calculated the percentages of cost increases on the basis of constant 2007 dollars to make them comparable across projects and to show real increases in cost while excluding increases due to inflation. [C] NNSA officials, in commenting on our draft report, stated that initial and current cost estimates for the Mixed Oxide Fuel Fabrication Facility and the Pit Disassembly and Conversion Facility should not be used in this analysis because neither project has an approved budget quality baseline. Nevertheless, we included the estimates in this analysis because both projects have been in an extended period of project design, without an approved budget-quality baseline, for about 10 years, and the estimates provided here are the only estimates available. [D] Estimate may change when DOE approves contractor's revised TPC in 2007. [End of table] In addition, as shown in table 2, 9 of the 12 projects experienced schedule delays ranging from 9 months to more than 11 years. Of the 9 projects, 7 had schedule delays of at least 2 years or more. Table 2: Changes in Estimated Project Schedules for DOE Major Construction Projects: Project: Pit Disassembly and Conversion Facility; Year mission need was approved: 1997; Initial completion date estimate: 06/2005; Current completion date estimate: 03/2017; Schedule delay as of February 2007: 11 years, 9 months. Project: Mixed Oxide Fuel Fabrication Facility; Year mission need was approved: 1997; Initial completion date estimate: 09/2004; Current completion date estimate: 03/2016; Schedule delay as of February 2007: 11 years, 6 months. Project: Waste Treatment and Immobilization Plant; Year mission need was approved: 1995; Initial completion date estimate: 07/2011; Current completion date estimate: 11/2019; Schedule delay as of February 2007: 8 years, 4 months. Project: National Ignition Facility; Year mission need was approved: 1993; Initial completion date estimate: 10/2003; Current completion date estimate: 03/2009; Schedule delay as of February 2007: 5 years, 5 months. Project: Depleted Uranium Hexafluoride 6 Conversion Facility[A]; Year mission need was approved: 2000; Initial completion date estimate: 03/ 2006; Current completion date estimate: 06/2008; Schedule delay as of February 2007: 2 years, 3 months. Project: Salt Waste Processing Facility; Year mission need was approved: 2001; Initial completion date estimate: 07/2009; Current completion date estimate: 09/2011[B]; Schedule delay as of February 2007: 2 years, 2 months. Project: Tritium Extraction Facility; Year mission need was approved: 1995; Initial completion date estimate: 06/2005; Current completion date estimate: 07/2007; Schedule delay as of February 2007: 2 years, 1 month. Project: Highly Enriched Uranium Materials Facility; Year mission need was approved: 1999; Initial completion date estimate: 04/2008; Current completion date estimate: 03/2010; Schedule delay as of February 2007: 1 year, 11 months. Project: Spallation Neutron Source; Year mission need was approved: 1996; Initial completion date estimate: 09/2005; Current completion date estimate: 06/2006[C]; Schedule delay as of February 2007: 9 months. Project: Chemistry and Metallurgy Research Facility Replacement; Year mission need was approved: 2002; Initial completion date estimate: 03/ 2014; Current completion date estimate: 03/2014; Schedule delay as of February 2007: Not applicable. Project: Microsystems and Engineering Sciences Applications; Year mission need was approved: 1999; Initial completion date estimate: 01/ 2009; Current completion date estimate: 01/2009; Schedule delay as of February 2007: Not applicable. Project: Linac Coherent Light Source; Year mission need was approved: 2001; Initial completion date estimate: 03/2009; Current completion date estimate: 03/2009; Schedule delay as of February 2007: Not applicable. Source: GAO analysis of DOE data. [A] This project reported a schedule delay but did not report an increase in the estimated total project cost (TPC). According to the DOE project director, the original cost estimate was probably too high and was not well supported. [B] According to DOE officials, schedule may slip further when the contractor submits its revised TPC to DOE in July 2007. [C] Project was completed on this date. Transition to operations has begun. [End of table] As table 3 shows, ineffective DOE project oversight and poor contractor management were frequently cited reasons for cost increases and schedule delays for the projects we reviewed, according to our review of independent studies of the 9 projects experiencing cost growth and schedule delays and our follow-up interviews with DOE project directors. Project officials, in commenting on our draft report, were concerned that table 3 might misrepresent the overall successful execution and completion of some projects, such as the Spallation Neutron Source, and that some problems may have already been addressed. Nevertheless, to clarify our main purpose for table 3, our intent is to show broad categories of major reasons for cost increases and schedule delays, regardless of when they occurred or whether they have been adequately addressed. Table 3: Reasons for Cost Increases and Schedule Delays: Project: Depleted Uranium Hexafluoride 6 Conversion Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): X. Project: Highly Enriched Uranium Materials Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): X. Project: Mixed Oxide Fuel Fabrication Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/ security): X. Project: National Ignition Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): [Empty]. Project: Pit Disassembly and Conversion Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): X. Project: Salt Waste Processing Facility; DOE project oversight: X; Poor contractor management: [Empty]; External factors (e.g., safety/ security): X. Project: Spallation Neutron Source; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): [Empty]. Project: Tritium Extraction Facility; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): X. Project: Waste Treatment and Immobilization Plant; DOE project oversight: X; Poor contractor management: X; External factors (e.g., safety/security): X. Project: Total; DOE project oversight: 9; Poor contractor management: 8; External factors (e.g., safety/security): 7. Source: GAO analysis of independent project studies and interviews with DOE project directors (a list of the project studies we reviewed is included in app. III). [End of table] The DOE project oversight issues mentioned in table 3 include the following: * inadequate systems for measuring contractor performance; * approval of construction activities before final designs were sufficiently complete; * ineffective project reviews; * insufficient DOE staffing and experience; * inadequate use of project management controls; * lack of headquarters assistance and oversight support of field project directors; * failure to detect contractor performance problems, including inadequate federal inspection activities; and: * poor government cost estimates, including inadequate funding for contingencies. DOE's lack of adequate systems to measure contractor performance was cited in a December 2005 DOE Inspector General review of the Mixed Oxide Fuel Fabrication Facility. The Inspector General criticized DOE's NNSA for failing to approve a baseline against which to measure contractor performance and relying on outdated cost plans.[Footnote 11] According to the report, NNSA relied on confusing and misleading information detailed in the monthly project reports to monitor progress and track costs--reports that the contractor acknowledged as being "useless for evaluating performance or managing the project." Furthermore, although the contractor reported unfavorable cost and schedule variances for months, these variances were inaccurate and meaningless because performance was being compared against a 2-year-old plan. NNSA, in commenting on our draft report, stated that project oversight and contractor management problems identified in previous GAO, Inspector General, and other independent assessments have led to extensive improvements to the project, and that major findings identified during a recent independent review have been successfully addressed. Similarly, DOE's approval of construction activities before final designs were sufficiently complete has contributed significantly to project cost growth and schedule delays. As we have previously reported, the accelerated fast-track, design-build approach used for the Waste Treatment and Immobilization Plant, a highly complex first- of-a-kind nuclear facility, resulted in significant cost increases and schedule delays.[Footnote 12] DOE also allowed the contractor on another project, the Tritium Extraction Facility, to begin construction before the final design was completed to meet schedule commitments. According to a 2002 DOE Inspector General report on the project,[Footnote 13] this revised acquisition strategy of simultaneous design and construction directly resulted in at least $12 million in project overruns. The contractor management issues mentioned in table 3 include the poor management of technological challenges, among other contractor performance issues, according to DOE project directors. Cost increases and schedule delays for 6 of the 9 projects were due in part to contractors' poor management of the development and integration of technologies used in project designs by, among other things, not accurately anticipating the cost and time that would be required to carry out the highly complex tasks involved.[Footnote 14] For example: * The National Ignition Facility had over $1 billion in cost overruns and years of schedule delays, in large part because of technology integration problems. The requirements for the National Ignition Facility--the use of 192 high-power laser beams focused on a single target in a "clean room" environment--had not been attempted before on such a large scale. According to the DOE project director, early incorrect assumptions about the original facility design and the amount of work necessary to integrate the technologies and assemble the technical components contributed to about half of the project's cost increases and schedule delays. * The design of the Mixed Oxide Fuel Fabrication Facility has presented technical challenges in adapting the design of a similar plant in France to the design needs of this project. Although the technological challenge related to adopting the process designs from the French designs was not the primary contributor to the project's cost increases and schedule delays, according to NNSA officials, it has affected the project's complexity. The basic technology--combining plutonium oxide with depleted uranium to form fuel assemblies for use in commercial power reactors--has been previously demonstrated in France. However, the DOE project director told us that the DOE facility design must, among other things, account for processing surplus weapon-grade plutonium, a different type of material than processed in the French facility, and must be adapted to satisfy U.S. regulatory and other local requirements. In addition, the DOE facility faced the technological challenge of reducing the scale of components used in the French facility. Although definitive cost estimates are not yet available, expected costs for completing this project have grown by about $3.3 billion since 2002, and the schedule has been extended by more than 11 years, in part because the contractor did not initially understand the project's complexity and underestimated the level of effort needed to complete the work. NNSA explained that the capability of the reference plants currently in operation in France, and by extension, the Mixed Oxide Fuel Fabrication Facility process design, is currently being demonstrated by several prototype fuel assemblies manufactured with weapon-grade plutonium oxide, which are currently being successfully used in a reactor in South Carolina. * For the Waste Treatment and Immobilization Plant, a technology application used on the project had not been tested before construction. Filters, widely used in the water treatment industry, were being designed for the project to concentrate and remove radioactive particles in liquid waste, a new application for the filters. Although tests are currently under way to demonstrate the effectiveness of this application, project officials conceded that these filters may still not be appropriate for the project. Other contractor performance problems are illustrated by two examples. First, DOE cited the contractor working on the Highly Enriched Uranium Materials Facility for inadequate quality assurance that resulted in concrete work that did not meet design specifications. The subsequent suspension of construction activities and rework added to the project's estimated cost and schedule. Second, the DOE project director of the Depleted Uranium Hexafluoride 6 Conversion Facility told us that the project was delayed 2 years because the contractor (1) did not have experience in government contracts, (2) underestimated the design effort needed, and (3) failed to properly integrate the operations of three separate organizations it managed. As table 3 shows, external factors were cited as also contributing to cost growth and schedule delays, such as additional work to implement requirements for higher levels of safety and security in project operations, among other things. For example, design rework for 4 of the projects occurred in response to external safety oversight recommendations by the Defense Nuclear Facilities Safety Board that large DOE construction projects meet a certain level of personnel safety, and that their designs be robust enough to withstand certain seismic events. In addition, owing to new security requirements implemented after September 11, 2001, project officials on the Highly Enriched Uranium Materials Facility had to redesign some aspects of the project to ensure that heightened security measures were addressed. While DOE faced additional requirements for safety and security, it did not always reach timely decisions on how to implement these requirements, which contributed significantly to cost increases and schedule delays for the Salt Waste Processing Facility. The DOE project director for this project told us the Defense Nuclear Facilities Safety Board had expressed concerns in June 2004, 5 months after the preliminary design was started, that the facility design might not ensure nuclear wastes would be adequately contained in the event of earthquakes. However, DOE did not decide how to address this concern until 17 months later, as the project continued to move forward with the existing project design. According to the project director, better and more timely discussions between site officials and headquarters to decide on the actions needed to adequately address these safety and security requirements might have hastened resolution of the problem, and up to 1 year of design rework might have been avoided. The delay, the director told us, added $180 million to the total project cost and extended the schedule by 26 months. In commenting on our draft report, EM officials noted that it is now requiring a more rigorous safety analysis earlier in the decision-making process. Other external factors also contributed significantly to cost increases and delays for 2 interrelated projects we reviewed--the Mixed Oxide Fuel Fabrication Facility and the Pit Disassembly and Conversion Facility. Project officials for these projects told us that 25 to 50 percent of the cost increases and over 70 percent of the schedule delays they experienced were the direct result of Office of Management and Budget funding constraints and restrictions resulting from international agreements with Russia. That is, work that is delayed to a subsequent year because of funding constraints and other work restrictions can delay project completion, which likely increases total project costs. Similarly, Office of Science officials, commenting on our draft report, stated that external factors caused the largest percentage cost increase and schedule delay for the Spallation Neutron Source, including a reduced level of funding appropriated at a time when project activities and costs were increasing considerably. However, congressional funding was reduced in fiscal year 2000 because of concerns about poor project oversight and management in the early stages of this project. DOE officials also explained that a now-defunct policy may have contributed to increased costs and delays for several projects we examined. Until 2000, DOE required contractors to prepare cost and schedule estimates early in the project, before preliminary designs were completed. These estimates were used to establish a baseline for measuring contractor performance and tracking any cost increases or schedule delays. However, these estimates often were based on early conceptual designs and, thus, were subject to significant change as more detailed designs were developed. To improve the reliability of these estimates, DOE issued a new order in October 2000 that required the preparation of a cost estimate range at the start of preliminary design, and delayed the requirement for a definitive cost and schedule baseline estimate until after the preliminary design was completed. Consequently, DOE officials explained, the new policy should result in improved estimates and a more accurate measure of cost and schedule performance. We also sent a survey to DOE project directors for all 12 projects asking them to identify key events that led to the greatest cost increases or schedule delays, and the major factors contributing to these key events. However, no individual factors were identified as being major contributors to the cost increases or schedule delays. In responding to our survey, DOE project directors cited several factors that affected changes in cost and schedule. However, when asked to rate the relative significance of these factors for their impact on cost and schedule changes, the project directors generally did not judge them to be significant contributors to the changes. The most frequently cited factors were: * an absence of open communication, mutual trust, and close coordination; * changes in "political will" during project execution (e.g., project changes resulting from political decisions, both internal and external to the project); * interruptions in project funding; and: * project managers' lack of adequate professional experience. (For detailed survey results covering these four factors, see app. IV.) In contrast to the cost increases and schedule delays incurred on most of the projects we reviewed, 3 projects had not yet experienced cost increases or schedule delays--Microsystems and Engineering Sciences Applications, the Linac Coherent Light Source, and the Chemistry and Metallurgy Research Facility Replacement. DOE project officials identified key conditions that they believed helped avoid those cost increases and delays. These conditions included: * active oversight--that is, the DOE project directors were never "blindsided" by contractor issues; * a lack of technological complexity; * an effective system to measure contractor performance; * reliable cost estimates; * effective communication with and integration of all stakeholders; and: * sustained leadership. However, we observed that the Linac Coherent Light Source and the Chemistry and Metallurgy Research Facility Replacement facilities are still in a relatively early stage in the project development process, and thus it may be too early to gauge the overall success of either project. Additionally, because none of these 3 projects are highly technologically complex, they may be less susceptible to the types of problems associated with other projects we reviewed that experienced cost increases and delays. DOE Does Not Consistently Measure Technology Readiness to Ensure That Critical Technologies Will Work as Intended before Construction Begins: Although DOE requires its final designs to be sufficiently complete before beginning construction, it has not systematically ensured that the critical technologies reflected in project designs are technologically ready. Recognizing that a lack of technology readiness can result in cost overruns and schedule delays, other federal agencies, such as NASA and DOD, have issued guidance for measuring and communicating technology readiness. DOE Does Not Consistently Assess Technology Readiness: Only 1 of the 5 projects we reviewed to determine how DOE ensures that project designs are sufficiently complete before construction-- projects that were approaching or had recently begun construction--had a systematic assessment of technology readiness to determine whether the project components would work individually or collectively as expected in the intended design.[Footnote 15] Specifically, the DOE project director for the Pit Disassembly and Conversion Facility systematically measured and assessed readiness levels for each critical component of the overall project.[Footnote 16] The assessment was based on a method developed by NASA, that is, rating each technology from 0 to 10 in terms of relative maturity. Because the project has not yet begun construction, we could not determine whether the technology readiness assessment has helped project managers to avoid cost increases or schedule delays during construction. However, according to DOE and contractor officials responsible for the project, the assessment helped focus management attention during project design on critical technologies that may require additional resources to ensure that they are sufficiently ready before construction begins. In reviewing the assessment, however, we noted that project officials had not updated the assessment tool for this project for over 3 years. DOE's project director acknowledged the delay in updating the assessment and responded that he plans to begin updating the assessment annually. The other 4 projects did not have systematic assessments of technological readiness. Therefore, the risk associated with the technology may not be clearly and consistently understood across all levels of management. Formally approving the project's cost and schedule estimates as accurate and complete, or proceeding into construction, without having clearly assessed evidence of technology readiness can result in cost overruns and schedule delays. DOE's experience with the Waste Treatment and Immobilization Plant is a case in point. Specifically, technology known as "pulse jet mixers"[Footnote 17] was used in the design of a subsystem intended to prepare radioactive material for processing. However, this technology had not been used previously in this application, and it did not work in tests as expected, even after construction had already begun. Consequently, DOE incurred about $225 million in redesign costs and over 1 year in schedule delays, according to the DOE project director. Over the past several years, we and others have stressed the importance of assessing technology readiness to complete projects successfully, while avoiding cost increases and schedule delays. Specifically, by 1999, we reported that organizations using best practices recognize that delaying the resolution of technology problems until production or construction can result in at least a 10-fold cost increase.[Footnote 18] Furthermore, we reported that delaying the resolution until after the start of production could increase costs by 100-fold. Reporting on similar concerns, the National Research Council has identified factors common to large construction projects--in the areas of cost, schedule, and scope--that help to ensure projects are completed successfully.[Footnote 19] Among key technical conditions for defining project scope, the council stated, is a project plan that is based on employing the best available, state-of-the-art technology, but not experimental or unproven technology. As such, employing a consistent, systematic method for measuring the extent to which technology is still experimental or unproven is of critical importance. An assessment of technology readiness is even more crucial at certain points in the life of a project--particularly as DOE decides to accept a project's (1) preliminary design and formally approve the project's cost and schedule estimates as accurate and complete and (2) final design as sufficiently complete so that resources can be committed toward procurement and construction. Proceeding through these critical decision points without a credible and complete technology readiness assessment can lead to problems later in the project. Specifically, if DOE proceeds with the project when technologies are not yet ready, there is less certainty that the technologies specified in the preliminary or final designs will work as intended. Project managers may then need to modify or replace these technologies to make them work properly, which can result in costly and time-consuming redesign work. Moreover, modifying the design of a facility after construction has already begun can be expensive and time consuming. First, changes to an already designed work plan are not necessarily subject to competition because the new work can occur through "change orders"--that is, modifications to existing contracts. These change orders can be expensive, according to DOE project directors. Second, worker productivity can be lost if, for example, extra downtime results from delays to interrelated construction work. Finally, tearing down and rebuilding items already constructed, such as concrete floors, walls, and doors, might be necessary to accommodate a design change. DOE's experience in the predecessor project to the Salt Waste Processing Facility--the In-Tank Precipitation (ITP) project process-- at the Savannah River Site illustrates the potential consequences of proceeding with technology that is not sufficiently ready. As we reported in 2000, the ITP project was selected in 1983 as the preferred method for separating highly radioactive material from 34 million gallons of liquid stored at the Savannah River site--a step considered necessary to effectively handle this large quantity of waste.[Footnote 20] A 1983 test using the ITP technology on a tank containing 500,000 gallons of waste resulted in a significant buildup of benzene--a highly explosive and hazardous compound. The buildup of benzene was more than the tank instruments could register. Nevertheless, project managers decided to proceed with the project. In 1985, DOE estimated that it would take about 3 years and $32 million to construct the ITP facility. After a number of delays, the ITP facility was constructed and began start-up operations in 1995, which were halted because of safety concerns about the amount of benzene that the facility generated. In 1998, after about a decade of delays and costs of almost $500 million, DOE suspended the project because it did not work as safely and efficiently as designed. This suspension put an effective remedy for treating high-level waste at the Savannah River Site years behind schedule. DOE then directed its contractor to begin a process to identify and select an alternative technology, which has developed into the current project intended to treat this waste--the Salt Waste Processing Facility project. In response to our concerns about the 4 projects without systematic assessments of technology readiness, DOE project directors explained that they have alternative methods for assessing readiness. They are required to submit a project execution plan, which includes an assessment of risks, including technological risks, and a plan for mitigating risks. They also rely upon independent reviews, including extensive design reviews, before making critical decisions to accept designs, and cost and schedule estimates, or to proceed with construction. For example, DOE's Office of Engineering and Construction Management formally reviews major projects in an effort to ensure that the designs are sufficiently complete to begin construction. Specifically, an external independent readiness review is performed, often using the services of various independent technical experts, that, at a minimum, is intended to verify the readiness of the project to proceed into construction or to identify remedial action. Finally, several DOE project directors stated that they intentionally have avoided using fast-track, design-build approaches because of the many problems it posed for the Waste Treatment and Immobilization Plant project. The DOE project directors of the 5 DOE projects that are nearing, or have recently begun construction, told us they have completed, or expect to complete prior to construction, 85 to 100 percent of their projects' final design. In addition to following the more standard approaches for managing projects, such as preparing risk assessment plans, some DOE offices have developed their own tools for assessing the readiness of projects. For example, DOE's Office of Environmental Management (EM) uses a Product Definition Rating Index (PDRI) as a tool to assess how well a project is planned, and whether it is ready to proceed to the next project phase. Project elements rated include cost, schedule, scope/ technical, management planning and control, and external factors. Among the 77 project elements rated, 2 involve technology--the identification of technology development requirements, and the testing and evaluation of the technology to be used. While the project technologies are collectively given a ranking with this tool, the PDRI does not represent a rigorous examination of the demonstrated readiness of each critical technology for its application in the project. Furthermore, not all EM projects we examined were using this tool. DOE's design reviews, risk assessments, and other actions to monitor design completion are extensive and certainly have merit. However, we found that these actions alone do not provide consistent and transparent assurance that all technologies are sufficiently ready because they do not use a consistent and systematic method of measurement. DOE's project design reviews, for example, do not always clearly distinguish between technology that has been demonstrated to work as expected in the intended design versus a judgment that the technology has potential for reaching a specific level of readiness. The external review of the technologies for the Mixed Oxide Fuel Fabrication Facility illustrates the shortfalls in DOE's current approach to assessing technology readiness and communicating the results of those assessments.[Footnote 21] The report concluded, among other things, that the method chosen by the contractor is the most rigorous and comprehensive, and should result in the most successful technology transfer possible. Furthermore, the review team was very impressed with the rigor with which designs and design changes were being managed, finding ample evidence verifying that the exact design process used by the French was being transferred to the United States facility. Although the external reviewers seemed to be impressed with many aspects of the design transfer, and concluded that the technologies should not be problematic, they had identified some key concerns about technology readiness in the body of their final report. The reviewers did not explain how they reconciled their conclusion with their concerns. To reconcile these differences, we obtained several clarifying statements from DOE's project director, technical experts, and one of the study's authors. These clarifying statements appear to support the reviewers' conclusions. However, without these statements, the level of technological readiness was not readily evident because the independent review lacked consistent, systematic criteria and a method for measuring the degree of readiness or clearly communicating assessment results, and the review was not transparent. DOE does not consistently assess technology readiness of project technologies because its project management guidance lacks comprehensive standards for systematically measuring and communicating the readiness of project technologies. Specifically, DOE lacks consistent metrics for determining technology readiness departmentwide, terminology to facilitate effective communication, and oversight protocols for reporting and reviewing technology readiness levels. DOE project management guidance is contained in two key documents--DOE Order 413.3A and Manual 413.3-1. Although the manual requires final designs to be sufficiently complete before beginning construction, it does not specify how technologies reflected in project designs are to be assessed for readiness--to determine that they have been sufficiently demonstrated to work as intended. Consequently, critical decisions made without standard measures are susceptible to varying interpretations of the actual technology readiness attained and the level needed for a project to proceed, which can easily vary among projects and among officials within a single project. Other Federal Agencies Use a Standard Method for Measuring and Communicating Technology Readiness: Other federal agencies have recognized the importance of ensuring that technologies have been sufficiently demonstrated for their intended purpose and have issued standard guidance for measuring and communicating TRLs. In particular, recognizing the need to measure the readiness level of project technologies, NASA began using a systematic method of measurement in the mid-1990s. NASA incorporated a structured TRL approach into guidance on integrated technology planning. Similarly, to improve DOD management of risk and technology development, the Deputy Under Secretary of Defense (Science and Technology) officially endorsed, in a July 2001 memorandum, the use of TRLs in new major programs. In 2002, DOD issued mandatory procedures for major defense acquisition programs and major automated information system acquisition programs, which identified technology readiness as a principal element of program risk. The procedures require the military services' science and technology officials to conduct a systematic assessment of critical technologies that are identified in major weapon systems programs before starting engineering and manufacturing development and production. Using TRLs is the preferred method, and approval must be obtained from the Deputy Under Secretary if an equivalent alternative method is used, according to the Deputy Under Secretary's memorandum. Importantly, the procedures stated that TRLs are a measure of demonstrated technical maturity--they do not discuss the probability of occurrence (i.e., the likelihood of attaining required maturity) or the impact of not achieving technology maturity. Both NASA and DOD use a nine-point scale to measure technology readiness, from a low of TRL 1 (basic principles observed) to a high of TRL 9 (total system used successfully in project operations). (App. V contains the definitions of these nine TRLs.) For example, a subsystem prototype that has been successfully demonstrated in an operational environment would receive a higher TRL value (i.e., TRL 7) than a technological component that has been demonstrated in a laboratory test (i.e., TRL 4). In our previous work, we recommended to the Secretary of Defense that key project technologies used in weapons systems be demonstrated in an operational environment, reaching a high maturity level--analogous to TRL 7--before deciding to commit to a cost, schedule, and performance baseline for development and production of the weapon system.[Footnote 22] In response to our recommendation, DOD has agreed that if a technology does not achieve a score of TRL 6 or 7, project managers must develop a plan to bring the technology to the required readiness level before proceeding to the next project phase. Use of TRLs is not by itself a cure-all for managing critical technologies, but TRLs can be used in conjunction with other measures to improve the way projects are managed. For example, according to studies by NASA, DOD, and others, TRLs can: * provide a common language among the technology developers, engineers who will adopt/use the technology, and other stakeholders; * improve stakeholder communication regarding technology development-- a by-product of the discussion among stakeholders that is needed to negotiate a TRL value; * reveal the gap between a technology's current readiness level and the readiness level needed for successful inclusion in the intended product; * identify at-risk technologies that need increased management attention or additional resources for technology development to initiate risk-reduction measures; and: * increase transparency of critical decisions by identifying key technologies that have been demonstrated to work or by highlighting still immature or unproven technologies that might result in high project risk. Two DOE headquarters offices have attempted to systematically assess technology readiness. First, under the Office of Nuclear Energy, a DOE contractor preparing a congressional report used a TRL method to compare the maturity of advanced fuel cycle technologies. In addition, in 2000, DOE's Office of Science and Technology, under EM, issued a report that defined a process for assessing technology maturity of EM projects.[Footnote 23] However, according to an EM official, the office decided to discontinue using this assessment process because it was considered overly burdensome. As a result, DOE devolved responsibility for managing technology readiness to the contractor level. According to several DOE project directors we spoke with, a consistent, systematic method for assessing technology readiness would help achieve a number of objectives: that is, standardize terminology, make technology assessments more transparent, and improve communication among project stakeholders before they make critical project decisions. DOE project managers also acknowledged that TRLs could improve project management departmentwide, and some managers are now attempting to use this tool to assess technology maturity. The DOE project director for the Waste Treatment and Immobilization Plant told us that a senior DOE official encouraged him to begin using TRLs. He is consulting with DOD officials knowledgeable about using the TRL method and expects to develop a TRL tool and have TRL determinations for major parts of the project in 2007. (App. VI compares DOD's product development process with DOE's project management process for major projects.) Conclusions: The magnitude of the cost increases and schedule delays for DOE's major projects is cause for serious rethinking of how DOE manages them. To its credit, DOE has completed, or expects to complete prior to construction, 85 to 100 percent of project design work for the 5 projects we reviewed that have recently begun or are nearing construction. However, DOE has not systematically addressed another key factor--the readiness level of the technologies it expects to use in these projects. DOE lacks comprehensive standards in DOE Order 413.3A and Manual 413.3-1 for systematically measuring and communicating the readiness of project technologies. Specifically, the department lacks consistent metrics for determining technology readiness departmentwide, terminology, and oversight protocols for reporting and reviewing TRLs. Without consistent measurement and communication of the readiness of technologies, DOE does not have a basis for defining the acceptable level of technological risk for each project, making critical decisions on accepting the validity of a project's total estimated cost and schedule, or proceeding with construction. Other federal agencies have recognized the need to consistently measure and communicate technology readiness to help avoid cost increases and delays that result from relying on immature technologies. DOD, for example, requires its managers to use a TRL process to measure technology readiness and generally requires a TRL 7 (as we had recommended) before system development and demonstration. In contrast, as DOE's poor track record for managing the technological complexity of major projects shows, DOE has not systematically measured the readiness of critical project technologies before it approves definitive cost and schedule estimates or begins construction. Furthermore, without a systematic method for measuring technological readiness, DOE cannot effectively communicate within the department and to the Congress whether projects are at risk of experiencing cost increases and schedule delays associated with technology problems. Recommendations for Executive Action: To improve decision making and oversight for major DOE construction projects, including how project technology readiness is measured and reported, we recommend that the Secretary of Energy evaluate and consider adopting a disciplined and consistent approach to assessing TRLs for projects with critical technologies that includes the following three actions: * Develop comprehensive standards for systematically measuring and communicating the readiness of project technologies. At a minimum, these standards should (1) specify consistent metrics for determining technology readiness departmentwide, (2) establish terminology that can be consistently applied across projects, and (3) detail the oversight protocols to be used in reporting and reviewing TRLs. In preparing these standards, DOE should consider lessons learned from NASA and DOD, and its own experience in measuring technology readiness. If DOE's evaluation results in the decision to adopt these standards, it should incorporate them into DOE Order 413.3A and Manual 413.3-1, and provide the appropriate training to ensure their proper implementation. * Direct DOE Acquisition Executives to ensure that projects with critical technologies reach a level of readiness commensurate with acceptable risk--analogous to TRL 7--before deciding to approve the preliminary design and commit to definitive cost and schedule estimates, and at least TRL 7 or, if possible, TRL 8 before committing to construction expenses. * Inform the appropriate committees and Members of Congress of any DOE decision to approve definitive cost and schedule estimates, or to begin construction, without first having ensured that project technologies are sufficiently ready (at TRL 7 or 8). This information should include specific plans for mitigating technology risks, such as developing backup technologies to offset the effects of a potential technology failure, and appropriate justification for accepting higher technological risk. Agency Comments and Our Evaluation: We provided a draft of this report to DOE for its review and comment. DOE's written comments are reproduced in appendix VII. DOE agreed with our recommendations but suggested revisions that would allow it to first conduct a pilot application on selected projects to better understand the technology readiness assessment process and evaluate its potential use. We revised our recommendations to give DOE this flexibility. DOE also provided detailed technical comments, which we have incorporated into our report as appropriate. DOE also expressed several specific concerns with our draft report. First, DOE stated that while our draft broadly asserts that DOE project management has led to increases in cost and schedule, our recommendations are narrowly focused on technology assessment. We agree that our draft states that DOE project management has led to cost increases and schedule delays, a conclusion we reached on the basis of our contact with DOE project directors and our review of numerous studies and reports on DOE major projects. Our recommendations address technology assessment, a critical project management activity, because they were developed primarily on the basis of our specific finding that DOE lacks a systematic approach to ensure that final project designs, including critical technologies reflected in these designs, have been demonstrated to work as intended prior to construction. This report explains that delaying resolution of technology problems until construction can potentially lead to significant cost increases and schedule delays. Second, DOE stated that our draft report inappropriately characterizes cost and schedule growth from a small sample of projects by using preliminary cost and schedule estimates that are intended for internal DOE planning. To clarify, the scope of our review included an evaluation of DOE's major construction projects. In addition, our report explains that DOE changed its project management policy in 2000 to allow cost and schedule estimates to be prepared later in the project--at the end of preliminary design. Prior to this new policy, project directors submitted cost and schedule estimates earlier in the project development phase--at the end of conceptual design. For projects under way prior to the policy in 2000, we used post-2000 validated baseline estimates, if available. Otherwise, we used earlier estimates since these were the only estimates available and had been previously used by DOE to inform Congress of the total expected project cost and schedule while seeking initial project funding. We also note that for the five projects that were started after the new policy in 2000, we used the validated project baseline estimates recommended by DOE, if available. Third, DOE suggested we revise table 3 in our report to more clearly identify the correlation between cost and schedule growth and technology maturity. As our report states, the information in table 3 was drawn from the results of our review of independent studies involving the projects we reviewed and the results of our interviews with DOE project directors. Our report explains that cost increases and schedule delays for 6 of the 9 projects shown in the table were due in part to contractors' poor management of the development and integration of technologies used in the project designs. Finally, DOE stated that it is unclear how the factors cited in appendix IV, such as communication, and changes in "political will," among other things, led to our recommendation to assess technology readiness. Although not all of the factors cited in our survey have a link to our recommendation on technology readiness, one factor in particular--absence of communication--is addressed in our recommendation. Specifically, we recommended that the Secretary of Energy consider developing comprehensive standards for systematically measuring and communicating the readiness of project technologies, including the establishment of terminology that is to be consistently applied across projects. We are sending copies of the report to interested congressional committees, the Secretary of Energy, and the Director of the Office of Management and Budget. We will make copies available to others on request. In addition, the report will also be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staffs have any questions about this report, please contact me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other staff contributing to the report are listed in appendix VIII. Signed by: Gene Aloise: Director, Natural Resources and Environment: [End of section] Appendix I: Scope and Methodology: To determine the extent to which the Department of Energy's (DOE) major construction projects have experienced cost increases and schedule delays and the factors that have contributed to these problems, we identified (1) active DOE major line-item construction projects that have current total project cost estimates above the $750 million threshold--DOE's criteria for "major construction projects," and (2) the projects with estimates above $400 million--the DOE threshold for major projects until July 2006. We also identified those projects above $300 million to account for any projects that may pass the $400 million threshold.[Footnote 24] In all, we identified the following 12 projects: * Five of these 12 projects began before DOE moved its requirement for firm cost and schedule estimates to later in the project: the National Ignition Facility, the Mixed Oxide Fuel Fabrication Facility, the Pit Disassembly and Conversion Facility, the Spallation Neutron Source, and the Tritium Extraction Facility. We used the estimates at the end of conceptual design, as reported by project directors, for the initial project cost and schedule estimates. * Four of the remaining 7 projects had cost and schedule estimates completed at the end of preliminary design, according to the new DOE guidelines: the Highly Enriched Uranium Materials Facility, Microsystems and Engineering Sciences Applications, the Depleted Uranium Hexafluoride 6 Conversion Facilities, and the Linac Coherent Light Source. For these projects, we considered the estimates as reported by project directors to be the initial cost and schedule estimates. * One project, the Waste Treatment and Immobilization Plant, began after DOE moved the requirement for firm cost and schedule estimates to later in the project. However, DOE initially exempted the contractor from submitting firm cost and schedule estimates. Therefore, we used the estimates reported by the project director to be the initial cost and schedule estimates. * The final 2 projects, although falling under the new DOE requirements, had yet to complete their preliminary design at the time of our review: the Chemistry and Metallurgy Research Facility Replacement and the Salt Waste Processing Facility. For these projects, we considered the cost and schedule estimates at the end of conceptual design reported by project directors to be the initial project cost and schedule estimates. Because we and others have previously expressed concern about the data reliability of a key DOE project management tracking database--the Project Assessment and Reporting System--we did not develop conclusions or findings based on information generated through that system.[Footnote 25] Instead, we collected information directly through surveys and interviews with project site officials. To identify cost increases and schedule delays, and the factors that may have contributed to these changes, we surveyed DOE project directors, interviewed DOE and contractor project personnel, and reviewed project management documents for 12 major projects. These 12 projects are managed by DOE's Office of Science, Office of Environmental Management (EM), or National Nuclear Security Administration (NNSA). (See app. II for information on these projects.) Our survey asked DOE project directors of the 12 projects to identify the degree to which cost and schedule estimates may have changed and the reasons for these changes, and to describe the events and conditions that led to any changes. Eight of the 12 project directors responded that their projects had experienced cost increases and schedule delays, and 1 project director reported only a schedule delay. For these 9 projects, we asked project directors to (1) identify the top three events that led to the cost and schedule delays and (2) indicate to what extent certain factors may have contributed to the event that led to the largest percentage cost increase or schedule delay. The factors included in the survey instrument were based on the results of a National Research Council study that listed essential or important conditions needed for the successful completion of major projects.[Footnote 26] We asked project directors to identify the extent to which the lack of these conditions may have contributed to any cost and schedule delays. (App. IV shows key survey results for these 9 projects.) In addition to reviewing project documentation, we conducted site visits for the 9 projects that had experienced cost and schedule changes, and we analyzed (1) studies of these projects completed by DOE's Office of Inspector General and (2) external independent project reviews conducted under the direction of DOE's Office of Engineering and Construction Management in Washington, D.C. We interviewed federal project directors of the 3 projects that had not experienced cost increases or schedule delays to obtain information on factors they believe are important in avoiding such increases. To determine the extent to which DOE ensures that project designs are sufficiently complete before construction, we obtained additional information from project directors on 5 projects that were approaching, or had recently begun, construction. During our review, we obtained information on the extent project designs were, or are expected to be, complete before beginning construction, and the actions DOE had taken to ensure technologies used in these designs are sufficiently ready to begin construction. For 2 of these 5 projects, we applied a tool we previously had used to assess DOD programs--the tool enables project directors to characterize the readiness level of each technology being developed for use in aircraft and other military applications. In addition, we spoke with officials from DOE program offices and DOE's Office of Engineering and Construction Management in Washington, D.C. We provided interim briefings to the Subcommittee on Energy and Water Development, House Committee on Appropriations, on the status of our work in May and September, 2006. We performed our work between December 2005 and January 2007, in accordance with generally accepted government auditing standards. [End of section] Appendix II: Information on the 12 Department of Energy Major Projects Reviewed: Table: Project: Chemistry and Metallurgy Research Facility Replacement; DOE program office: National Nuclear Security Administration; Project purpose/objectives: Relocate and consolidate mission-critical analytical chemistry, material characterization, and research and development capabilities to ensure continuous national security mission support beyond 2010. Project: Depleted Uranium Hexafluoride 6 Conversion Facility; DOE program office: Office of Environmental Management; Project purpose/ objectives: Design and construct facilities at Portsmouth, Ohio, and Paducah, Kentucky, to convert the Department of Energy's existing inventory of depleted uranium hexafluoride into a more stable form for disposal or beneficial reuse. Project: Highly Enriched Uranium Materials Facility; DOE program office: National Nuclear Security Administration; Project purpose/ objectives: Project will construct a highly secure, state-of-the-art facility for consolidating and storing highly enriched uranium, resulting in cost savings and an increased security posture. Project: Linac Coherent Light Source; DOE program office: Science; Project purpose/objectives: Provide laser-like radiation in the X-ray region of the spectrum that is 10 billion times greater in peak brightness than any existing X-ray light source. The project will apply these high-brightness X-rays to experiments in the chemical, material, and biological sciences. Project: Microsystems and Engineering Sciences Applications; DOE program office: National Nuclear Security Administration; Project purpose/objectives: Provide state-of-the-art national complex that will provide for the design, integration, prototyping, and qualification of microsystems into components, subsystems, and systems within the nuclear weapons stockpile. Project: Mixed Oxide Fuel Fabrication Facility; DOE program office: National Nuclear Security Administration; Project purpose/objectives: Facility will combine surplus weapon-grade plutonium oxide with depleted uranium to form mixed oxide fuel assemblies that will be irradiated in United States commercial nuclear reactors. Once irradiated and converted into spent fuel, the resulting plutonium can no longer be readily used for nuclear weapons. Project: National Ignition Facility; DOE program office: National Nuclear Security Administration; Project purpose/objectives: Provide experimental capability to assess nuclear weapons physics, providing critical data that will allow the United States to maintain its technical capabilities in nuclear weapons in the absence of underground testing, and to advance fusion as an energy source. Project: Pit Disassembly and Conversion Facility; DOE program office: National Nuclear Security Administration; Project purpose/objectives: Eliminate surplus Russian and United States plutonium and highly enriched uranium by disassembling surplus nuclear weapons pits and converting the resulting plutonium metal to a powder form that can later be fabricated into mixed oxide fuel to produce nuclear fuel assemblies for use in commercial nuclear reactors. Project: Salt Waste Processing Facility; DOE program office: Office of Environmental Management; Project purpose/objectives: Meet site cleanup goals and reduce significant environmental and health/safety risk by construction of a facility to treat large quantities of waste from reprocessing and nuclear materials production operations at the Savannah River Site. Process will separate waste, solidify it in glass, and send it to federal repositories for disposal. Project: Spallation Neutron Source; DOE program office: Science; Project purpose/objectives: Provide next generation, short-pulse spallation neutron source for neutron scattering, to be used by researchers from academia, national and federal labs, and industry for basic and applied research and technology development in the fields of condensed matter physics, materials sciences, magnetic materials, polymers and complex fluids, chemistry, biology, earth sciences, and engineering. Project: Tritium Extraction Facility; DOE program office: National Nuclear Security Administration; Project purpose/objectives: To replenish the tritium needs of the nuclear weapons stockpile, the facility will extract tritium produced in a commercial nuclear reactor for use in nuclear weapons development. Project: Waste Treatment and Immobilization Plant; DOE program office: Office of Environmental Management; Project purpose/objectives: The plant will separate high-level from low- level radioactive waste currently stored in underground tanks, processing and solidifying all high-level waste and a substantial portion of the low-level waste, and will treat the remaining low-level waste. Source: DOE. [End of table] [End of section] Appendix III: Independent Studies Reviewed: National Ignition Facility: Department of Energy, Office of Inspector General. Audit Report: Status of the National Ignition Facility Project. DOE/IG-0598. Washington, D.C.: April 28, 2003. GAO. Department of Energy: Status of Contract and Project Management Reforms. GAO-03-570T. Washington, D.C.: March 20, 2003. GAO. Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results. GAO-02-798. Washington, D.C.: September 13, 2002. GAO. Department of Energy: Follow-up Review of DOE's National Ignition Facility. GAO-01-677R. Washington, D.C.: June 1, 2001. GAO. National Ignition Facility: Management and Oversight Failures Caused Major Cost Overruns and Schedule Delays. GAO/RCED-00-141 and GAO/RCED-00-271. Washington, D.C.: August 8, 2000. The Mitre Corporation. NIF Ignition. JSR-05-340. McLean, VA: June 29, 2005. Mixed Oxide Fuel Fabrication Facility: Burns and Roe Enterprises, Inc. External Independent Review of the Mixed Oxide Fuel Fabrication Facility (MFFF) Project Critical Decision (CD) 2/3 Baseline: Performance Baseline (CD-2) and Start of Construction (CD-3) Review. BREI-L-R-06-03. Oradell, NJ: July 7, 2006. Burns and Roe Enterprises, Inc. External Independent Review of the Basis of Design for the Aqueuous Polishing Process. BREI-SLP-R-06-01. Oradell, NJ: March 27, 2006. Civil Engineering Research Foundation. Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success. Washington, D.C.: July 12, 2004. Department of Energy, Office of Inspector General. Audit Report: Status of the Mixed Oxide Fuel Fabrication Facility. DOE/IG-0713. Washington, D.C.: December 21, 2005. Pit Disassembly and Conversion Facility: Department of Energy, Office of Inspector General. Audit Report: National Nuclear Security Administration's Pit Disassembly and Conversion Facility. DOE/IG-0688. Washington, D.C.: May 3, 2005. Los Alamos National Laboratory. Options for the Development and Testing of the Pit Disassembly and Conversion Facility Government-Furnished Design. LA-UR-03-3926. Los Alamos, NM: June 11, 2003. Waste Treatment and Immobilization Plant: Bechtel National, Inc. Hanford Tank Waste Treatment and Immobilization Plant, May 2006 Estimate at Completion. Hanford Site, WA: May 31, 2006. Bechtel National, Inc. Comprehensive Review of the Hanford Tank Waste Treatment and Immobilization Plant Estimate at Completion. CCN 132848. Hanford Site, WA: March 31, 2006. Bechtel National, Inc. Comprehensive Review of the Hanford Waste Treatment Plant Flowsheet and Throughput. CCN132846. Hanford Site, WA: March 17, 2006. Bechtel National, Inc. Hanford Tank Waste Treatment and Immobilization Plant, December 2005 Estimate at Completion Executive Summary. Hanford Site, WA: January 30, 2006. Department of the Army Corp of Engineers. Complete Statement of Kim Callan, to the Subcommittee on Energy and Water Development, Committee on Appropriations, United States House of Representatives. Washington, D.C.: April 6, 2006. Department of Energy. External Independent Review, Independent Cost Review, CD-3C Review of the Waste Treatment and Immobilization Plant Project. Hanford Site, WA: September 2002. Department of Energy. External Independent Review CD-3B Review of the Waste Treatment and Immobilization Plant Project. Hanford Site, WA: April 2002. GAO. Hanford Waste Treatment Plant, Contractor and DOE Management Problems Have Led to Higher Costs, Construction Delays, and Safety Concerns. GAO-06-602T. Washington, D.C.: April 6, 2006. GAO. Further Actions Are Needed to Strengthen Contract Management for Major Projects. GAO-05-123. Washington, D.C.: March 18, 2005. GAO. Nuclear Waste: Absence of Key Management Reforms on Hanford's Cleanup Project Adds to Challenges of Achieving Cost and Schedule Goals. GAO-04-611. Washington, D.C.: June 9, 2004. GAO. Status of Contract and Project Management Reforms. GAO-03-57T. Washington, D.C.: March 20, 2003. GAO. Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results. GAO-02-798. Washington, D.C.: September 13, 2002. GAO. Nuclear Waste: Hanford Tank Waste Program Needs Cost, Schedule, and Management Changes. GAO/RCED-93-99. Washington, D.C.: March 8, 1993. LMI Government Consulting. Hanford Waste Treatment and Immobilization Plant After-Action Fact-Finding Review. DE535T1. McLean, VA: January 2006. LMI Government Consulting. External Independent Review, Follow-up Review, Waste Treatment and Immobilization Plant (WTP) Out Briefing. Washington, D.C.: March 14, 2003. Spallation Neutron Source: Civil Engineering Research Foundation. Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success. Washington, D.C.: July 12, 2004. Department of Energy, Office of Inspector General. Audit Report: Progress of the Spallation Neutron Source Project. DOE/IG-0532. Washington, D.C.: November 19, 2001. Department of Energy. Review Committee Report on the Baseline Review of the Spallation Neutron Source (SNS) Project. Washington, D.C.: July 15, 1999. Department of Energy. Technical, Cost, Schedule, and Management Review of the Spallation Neutron Source Project. Washington, D.C.: January 28, 1999. GAO. Department of Energy: Status of Contract and Project Management Reforms. GAO-03-570T. Washington, D.C.: March 20, 2003. GAO. Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results. GAO-02-798. Washington, D.C.: September 13, 2002. GAO. Department of Energy: Challenges Exist in Managing the Spallation Neutron Source Project. GAO/T-RCED-99-103. Washington, D.C.: March 3, 1999. Salt Waste Processing Facility: Department of Energy, Office of Inspector General. Audit Report: Salt Processing Project at the Savannah River Site. DOE/IG-0565. Washington, D.C.: August 27, 2002. Institute for Regulatory Science. Technical Peer Review Report of the Review Panel on Salt Waste Processing Facility Technology Readiness. CRTD-Vol. 75. Danvers, MA: October 31, 2003. Tritium Extraction Facility: Civil Engineering Research Foundation. Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success. Washington, D.C.: July 12, 2004. Department of Energy, Office of Inspector General. Audit Report: The Department of Energy's Tritium Extraction Facility. DOE/IG-0560. Washington, D.C.: June 24, 2002. GAO. Department of Energy: Further Actions Are Needed to Strengthen Contract Management for Major Projects. GAO-05-123. Washington, D.C.: March 18, 2005. GAO. Department of Energy: Status of Contract and Project Management Reforms. GAO-03-570T. Washington, D.C.: March 20, 2003. GAO. Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results. GAO-02-798. Washington, D.C.: September 13, 2002. GAO. Nuclear Weapons: Design Reviews of DOE's Tritium Extraction Facility. GAO/RCED-98-75. Washington, D.C.: March 31, 1998. National Nuclear Security Administration. Program Review of the Estimate to Complete Tritium Extraction Facility (TEF) at Savannah River Site. Washington, D.C.: August 29, 2002. Highly Enriched Uranium Materials Facility: BWXT Y-12. Highly Enriched Uranium Materials Facility Project Causal Analysis Report. Oak Ridge, TN: March 6, 2006. Department of Energy. Limited External Independent Review for Baseline Change Proposal Review. Oak Ridge, TN: August 31, 2004. Department of Energy, Office of Inspector General. Audit Report, Design of the Uranium Storage Facility at the Y-12 National Security Complex. DOE/IG-0643. Washington, D.C.: March 19, 2004. Department of Energy, Office of Inspector General. Audit Report, Reestablishment of Enriched Uranium Operations at the Y-12 National Security Complex. DOE/IG-0640. Washington, D.C.: February 24, 2004. Department of Energy. External Independent Review - Performance Baseline Review of the Highly Enriched Uranium Materials Facility Project. Oak Ridge, TN: June 2003. Depleted Uranium Hexafluoride 6 Conversion Facility: Department of Energy. Report on the Independent Project Review of the Depleted Uranium Hexafluoride Conversion Project. Washington, D.C.: October 8, 2004. Department of Energy, Office of Inspector General. Audit Report: Depleted Uranium Hexafluoride Conversion. DOE/IG-0642. Washington, D.C.: March 18, 2004. GAO. Department of Energy: Status of Contract and Project Management Reforms. GAO-03-570T. Washington, D.C.: March 20, 2003. LMI Government Consulting. DUF6 Conversion Project CD-3 Corrective Action Plan Review. DE538T1. McLean, VA: October 2005. LMI Government Consulting. Construction Readiness EIR (for CD-3) of the Depleted Uranium Hexafluoride Conversion Project. DE534T1. McLean, VA: June 2005. LMI Government Consulting. DUF6 Conversion Project CD-3C Construction Readiness Review Preliminary Draft. Washington, D.C.: May 20, 2005. LMI Government Consulting. DUF6 Limited Conversion Plan Project External Independent Review for the Office of Engineering and Construction Management. DE428T1. McLean, VA: June 2004. Chemistry and Metallurgy Research Facility Replacement: Jupiter Corporation. External Independent Review of the Chemistry and Metallurgy Research Building Replacement Project. Approve Performance Baseline and Approve Start of Construction. CD-2A/CD-3A. Wheaton, MD: October 14, 2005. [End of section] Appendix IV: Survey Results for Primary Factors Affecting Cost and Schedule on Nine Projects with Cost or Schedule Changes: Table: Absence of open communication, mutual trust, and close coordination. Factor/Project: Depleted Uranium Hexafluoride 6 Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Highly Enriched Uranium Materials Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Mixed Oxide Fuel Fabrication Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: National Ignition Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Pit Disassembly and Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: Salt Waste Processing Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Spallation Neutron Source; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Tritium Extraction Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: X; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Waste Treatment and Immobilization Plant; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Total; Survey results for primary factors: To no extent: 0; Survey results for primary factors: To a limited extent: 4; Survey results for primary factors: To a moderate extent: 2; Survey results for primary factors: To a great extent: 1; Survey results for primary factors: To a very great extent: 0; Survey results for primary factors: No answer: 2. Changes in "political will" during project execution (e.g., project changes resulting from political decisions--includes politics internal and external to the project ). Factor/Project: Depleted Uranium Hexafluoride 6 Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Highly Enriched Uranium Materials Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Mixed Oxide Fuel Fabrication Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: National Ignition Facility; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Pit Disassembly and Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: X; Survey results for primary factors: No answer: [Empty]. Factor/Project: Salt Waste Processing Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Spallation Neutron Source; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: X; Survey results for primary factors: No answer: [Empty]. Factor/Project: Tritium Extraction Facility; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Waste Treatment and Immobilization Plant; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Total; Survey results for primary factors: To no extent: 2; Survey results for primary factors: To a limited extent: 2; Survey results for primary factors: To a moderate extent: 2; Survey results for primary factors: To a great extent: 0; Survey results for primary factors: To a very great extent: 2; Survey results for primary factors: No answer: 1. Interruptions in planning and committing budget funds. Factor/Project: Depleted Uranium Hexafluoride 6 Conversion; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Highly Enriched Uranium Materials Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: X; Survey results for primary factors: No answer: [Empty]. Factor/Project: Mixed Oxide Fuel Fabrication Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: X; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: National Ignition Facility; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Pit Disassembly and Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: Salt Waste Processing Facility; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Spallation Neutron Source; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: X; Survey results for primary factors: No answer: [Empty]. Factor/Project: Tritium Extraction Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Waste Treatment and Immobilization Plant; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: X; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Total; Survey results for primary factors: To no extent: 3; Survey results for primary factors: To a limited extent: 0; Survey results for primary factors: To a moderate extent: 1; Survey results for primary factors: To a great extent: 2; Survey results for primary factors: To a very great extent: 2; Survey results for primary factors: No answer: 1. Project managers did not have adequate professional experience. Factor/Project: Depleted Uranium Hexafluoride 6 Conversion; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Highly Enriched Uranium Materials Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Mixed Oxide Fuel Fabrication Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: National Ignition Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Salt Waste Processing Facility; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Spallation Neutron Source; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: X; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Tritium Extraction Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: X; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Pit Disassembly and Conversion Facility; Survey results for primary factors: To no extent: [Empty]; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: X. Factor/Project: Waste Treatment and Immobilization Plant; Survey results for primary factors: To no extent: X; Survey results for primary factors: To a limited extent: [Empty]; Survey results for primary factors: To a moderate extent: [Empty]; Survey results for primary factors: To a great extent: [Empty]; Survey results for primary factors: To a very great extent: [Empty]; Survey results for primary factors: No answer: [Empty]. Factor/Project: Total; Survey results for primary factors: To no extent: 2; Survey results for primary factors: To a limited extent: 1; Survey results for primary factors: To a moderate extent: 4; Survey results for primary factors: To a great extent: 0; Survey results for primary factors: To a very great extent: 0; Survey results for primary factors: No answer: 2. Source: GAO. [End of table] [End of section] Appendix V: Definitions of Technology Readiness Levels: Table: Technology readiness level(TRL): 1. Basic principles observed and reported; Level involved: Studies; Basic objective of TRLs: Research to prove feasibility; Components: None; Integration: None; Tests and Environment: Desktop "back of envelope" environment. Technology readiness level(TRL): 2. Technology concept and/or application formulated; Level involved: Studies; Basic objective of TRLs: Research to prove feasibility; Components: None; Integration: Paper studies indicate components ought to work together; Tests and Environment: Academic environment. The emphasis here is still on understanding the science but beginning to think about possible applications of the scientific principles. Technology readiness level(TRL): 3. Analytical and experimental critical function and/or characteristic proof or concept; Level involved: Pieces of components; Basic objective of TRLs: Research to prove feasibility; Components: No system components, just basic laboratory research equipment to verify physical principles; Integration: No attempt at integration; still trying to see whether individual parts of the technology work. Lab experiments with available components show they will work; Tests and Environment: Uses of the observed properties are postulated and experimentation with potential elements of subsystem begins. Lab work to validate pieces of technology without trying to integrate. Emphasis is on validating the predictions made during earlier analytical studies to ensure that the technology has a firm scientific underpinning. Technology readiness level(TRL): 4. Component and/or breadboard validation in lab environment; Level involved: Low fidelity breadboard; Basic objective of TRLs: Demonstrate technical feasibility and functionality; Components: Ad Hoc and available laboratory components are surrogates for system components that may require special handling, calibration, or alignment to get them to function. Not fully functional but representative of technically feasible approach; Integration: Available components assembled into subsystem breadboard. Interfaces between components are realistic; Tests and Environment: Tests in controlled laboratory environment. Lab work at less than full subsystem integration, although starting to see if components will work together. Technology readiness level(TRL): 5. Component and/or breadboard validation in relevant environment; Level involved: High fidelity breadboard/brass-board (e.g., nonscale or form components); Basic objective of TRLs: Demonstrate technical feasibility and functionality; Components: Fidelity of components and interfaces are improved from TRL 4. Some special purpose components combined with available laboratory components. Functionally equivalent but not of same material or size. May include integration of several components with reasonably realistic support elements to demonstrate functionality; Integration: Fidelity of subsystem mock up improves (e.g., from breadboard to brassboard). Integration issues become defined; Tests and Environment: Laboratory environment modified to approximate operational environment. Increases in accuracy of the controlled environment in which it is tested. Technology readiness level(TRL): 6. System/Subsystem model or prototype demonstration in relevant environment; Level involved: Subsystem closely configured for intended project application. Demonstrated in relevant environment. (Shows will work in desired configuration); Basic objective of TRLs: Demonstrate applicability to intended project and subsystem integration. (Specific to intended application in project.); Components: Subsystem is high fidelity functional prototype with (very near same material and size of operational system). Probably includes the integration of many new components and realistic supporting elements/subsystems if needed to demonstrate full functionality. Partially integrated with existing systems; Integration: Components are functionally compatible (and very near same material and size of operational system). Component integration into system is demonstrated; Tests and Environment: Relevant environment inside or outside the laboratory, but not the eventual operating environment. The testing environment does not reach the level of an operational environment, although moving out of controlled laboratory environment into something more closely approximating the realities of technology’s intended use. Technology readiness level(TRL): 7. Subsystem prototype demonstration in an operational environment; Level involved: Subsystem configured for intended project application. Demonstrated in operational environment; Basic objective of TRLs: Demonstrate applicability to intended project and subsystem integration. (Specific to intended application in project.); Components: Prototype improves to preproduction quality. Components are representative of project components (material, size, and function) and integrated with other key supporting elements/subsystems to demonstrate full functionality. Accurate enough representation to expect only minor design changes; Integration: Prototype not integrated into intended system but onto surrogate system; Tests and Environment: Operational environment, but not the eventual environment. Operational testing of system in representational environment. Prototype will be exposed to the true operational environment on a surrogate platform, demonstrator, or test bed. Technology readiness level(TRL): 8. Total system completed, tested, and fully demonstrated; Level involved: Full integration of subsystems to show total system will meet requirements; Basic objective of TRLs: Applied/Integrated into intended project application; Components: Components are right material, size, and function compatible with operational system; Integration: Subsystem performance meets intended application and is fully integrated into total system; Tests and Environment: Demonstration, test, and evaluation completed. Demonstrates system meets procurement specifications. Demonstrated in eventual environment. Technology readiness level(TRL): 9. Total system used successfully in project operations; Level involved: System meeting intended operational requirements; Basic objective of TRLs: Applied/Integrated into intended project application; Components: Components are successfully performing in the actual environment—proper size, material, and function; Integration: Subsystem has been installed and successfully deployed in project systems; Tests and Environment: Operational testing and evaluation completed. Demonstrates that system is capable of meeting all mission requirements. Source: GAO analysis of DOD data. [End of table] [End of section] Appendix VI: Comparison of DOD's Product Development Process with DOE's Project Management Process: [See PDF for Image] Source: GAO analysis of DOD and DOE data. [End of section] Appendix VII: Comments from the Department of Energy: Department of Energy: Washington, DC 20585: Mar 7 2007: Mr. Gene Aloise: Director, Natural Resources and Environment: U.S. Government Accountability Office: 441 G Street NW: Washington, DC 20548: Dear Mr. Aloise: The Department of Energy (DOE or Department) has reviewed the draft Government Accountability Office (GAO) report entitled "Major Construction Projects Need a Consistent Approach for Assessing Technology Readiness to Help Avoid Cost Increases and Delays" (GAO-07- 336). The report asserts broadly that the Department's management of projects has led to growth in cost and schedules, but the report's recommendations focus only on implementing a technology assessment process. The Department agrees that appropriately assessing technology readiness can be a significant factor in successfully completing its projects. However, the Department has concerns with (1) the manner in which GAO characterizes cost and schedule growth from a small sample of our projects, and (2) the establishment of a standard for evaluating (and approving) technology readiness for all DOE projects without appropriate study. The report inappropriately characterizes the cost growth associated with DOE projects by using preliminary cost and schedule estimates intended for internal DOE planning rather than validated and approved baselines. These cost and schedule baselines are established at Critical Decision-2 after having been validated by the Department's Office of Engineering and Construction Management and approved by the applicable Acquisition Executive. Tables 1 and 2, which compare inconsistent combinations of preliminary and validated data, should be deleted from the report or revised using only validated data for initial and current baselines to allow for an accurate comparison. The findings and conclusions supported by these tables should also be corrected. While the report caveats the basis for cost and schedule growth in the tables, the explanation provided is not sufficient to prevent readers from reaching the wrong conclusions regarding cost and schedule growth for the sample of projects in the study. The report also attempts to identify the causes for the cost and schedule increases in Table 3. The information contained in Table 3 should be revised to more clearly identify the correlation between cost and schedule growth and technology maturity. Additionally, GAO's data in Appendix IV, Survey Results, focuses on communication, changes in "political will", funding uncertainties, and project manager experience. It is unclear how these factors lead to the recommendation to assess technology readiness. Despite concerns with the study and report, the Department believes GAO's recommendation regarding technology readiness analysis has merit. The Department plans to pilot application of the technology readiness assessment process on selected projects in order to better understand the process and evaluate its potential use in its diverse portfolio of projects. If warranted, DOE could implement a consistent technology readiness assessment as part of its established project management system. The assessment process would likely be applied to projects on a case by case basis where critical technologies are being used. The Department suggests that GAO's recommendation be restated as follows: As one tool to improve project planning and execution, we recommend that the Secretary of Energy evaluate the use of a technology readiness assessment for projects that involve critical technologies. * The Department of Energy should consider lessons learned from NASA and DoD on the use of Technology Readiness Levels (TRLs), as well as the Department's own experience in determining technology readiness, as it evaluates protocols for assessing and communicating the readiness of critical project technologies. * The Acquisition Executive should consider project risk imposed by immature technology, and on a case by case basis, define an appropriate level of technology readiness prior to establishing the project's baseline. Additional project-specific comments and corrections are attached. The Department requests that this response letter be included in GAO's final report. Sincerely, Signed by: Ingrid Kolb: Director: Office of Management: Attachment: [End of section] Appendix VIII: GAO Contact and Staff Acknowledgments: GAO Contact: Gene Aloise, (202) 512-3841: Staff Acknowledgments: In addition to the individual named above, Michaela Brown, Rudy Chatlos, James Espinoza, Daniel Feehan (Assistant Director), Joseph Keener, Thomas Kingham, Matthew Lea, Mehrzad Nadji, Omari Norman, Christopher Pacheco, Thomas Perry, and Carol Herrnstadt Shulman made key contributions to this report. FOOTNOTES [1] For this review, we lowered the threshold to $300 million out of concern that some projects not considered major could later be defined as major because of cost increases. [2] This estimate includes design and construction costs, but does not reflect the total life-cycle costs of the projects, such as operating and maintenance costs. [3] GAO, Hanford Waste Treatment Plant: Contractor and DOE Management Problems Have Led to Higher Costs, Construction Delays, and Safety Concerns, GAO-06-602T (Washington, D.C.: Apr. 6, 2006). [4] A forthcoming GAO report will address actions taken by DOE to improve overall project management. [5] GAO, Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999); Defense Acquisitions: Assessments of Selected Major Weapons Programs, GAO-06-391 (Washington, D.C.: Mar. 31, 2006); and Defense Acquisitions: Space-Based Radar Effort Needs Additional Knowledge before Starting Development, GAO-04-759 (Washington, D.C.: July 23, 2004). [6] GAO, Department of Energy: Further Actions Are Needed to Strengthen Contract Management for Major Projects, GAO-05-123 (Washington, D.C.: Mar. 18, 2005); and Civil Engineering Research Foundation, Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success (Washington, D.C: July 12, 2004). [7] GAO, Oversight of DOE's Major Systems, GAO/RCED-97-146R (Washington, D.C.: Apr. 30, 1997). [8] The National Research Council was organized by the National Academy of Sciences to advise the federal government on matters related to science and technology. [9] National Research Council, Improving Project Management in the Department of Energy (Washington, D.C.: July 1999). [10] Civil Engineering Research Foundation, Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success (Washington, D.C.: July 12, 2004). [11] Department of Energy, Office of Inspector General, Audit Report: Status of the Mixed Oxide Fuel Fabrication Facility, DOE/IG-0713 (Washington, D.C.: December 2005). [12] GAO-06-602T. [13] Department of Energy, Office of Inspector General, Audit Report: The Department of Energy's Tritium Extraction Facility, DOE/IG-0560 (Washington, D.C.: June 2002). [14] These 6 projects are the Mixed Oxide Fuel Fabrication Facility, National Ignition Facility, Pit Disassembly and Conversion Facility, Spallation Neutron Source, Tritium Extraction Facility, and Waste Treatment and Immobilization Plant. [15] These 5 projects are the Chemistry and Metallurgy Research Facility Replacement, Depleted Uranium Hexafluoride 6 Conversion Facility, Mixed Oxide Fuel Fabrication Facility, Pit Disassembly and Conversion Facility, and Salt Waste Processing Facility. [16] Los Alamos National Laboratory, Options for the Development and Testing of the Pit Disassembly and Conversion Facility Government- Furnished Design, LA-UR-03-3926 (Los Alamos, New Mexico: June 11, 2003). [17] Pulse jet mixers, which do not have moving parts, use compressed air to continuously mix tank waste so that it can be properly prepared for further processing. While such devices have previously been used successfully in other applications, they have never been used for mixing wastes with high-solid content like those at the Waste Treatment and Immobilization Plant. [18] GAO/NSIAD-99-162 and GAO, Joint Strike Fighter Acquisition: Mature Critical Technologies Needed to Reduce Risks, GAO-02-39 (Washington, D.C.: Oct. 19, 2001). [19] Improving Project Management. [20] GAO, Department of Energy: Uncertainties and Management Problems Have Hindered Cleanup at Two Nuclear Waste Sites, GAO/T-RCED-00-248 (Washington, D.C.: July 12, 2000). [21] Burns and Roe Enterprises, Inc., External Independent Review of the Basis of Design for the Aqueous Polishing Process for the Mixed Oxide Fuel Fabrication Facility at The Savannah River Site for the U.S. Department of Energy Office of Engineering and Construction Management and National Energy Technology Laboratory Report, BREI-LSP-R-06-01 (Oradell, New Jersey: March 2006). [22] GAO/NSIAD-99-162. [23] Department of Energy, Tracking Technology Maturity in DOE's Environmental Management Science and Technology Program; Revision 1 (Washington, D.C.: Jan. 1, 2001). [24] We excluded the Yucca Mountain Repository project, with a total estimated cost of $23 billion, from our review due to its uniqueness and the fact that we have recently reported on the project and currently have an ongoing review. Also, to review projects with sufficient maturity, we included only the projects that were at least 1 year past completion of conceptual design. [25] GAO, Department of Energy: Further Actions Are Needed to Strengthen Contract Management for Major Projects, GAO-05-123 (Washington, D.C.: Mar. 18, 2005); and Civil Engineering Research Foundation, Independent Research Assessment of Project Management Factors Affecting Department of Energy Project Success (Washington, D.C.: July 12, 2004). [26] National Research Council, Improving Project Management in the Department of Energy (Washington, D.C.: July 1999). GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: