This is the accessible text file for GAO report number GAO-06-313 
entitled 'Yucca Mountain: Quality Assurance at DOE's Planned Nuclear 
Waste Repository Needs Increased Management Attention' which was 
released on March 23, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Subcommittee on the Federal Workforce and 
Agency Organization, Committee on Government Reform, House of 
Representatives: 

March 2006: 

Yucca Mountain: 

Quality Assurance at DOE's Planned Nuclear Waste Repository Needs 
Increased Management Attention: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-313]: 

GAO Highlights: 

Highlights of GAO-06-313, a report to the Chairman, Subcommittee on the 
Federal Workforce and Agency Organization, Committee on Government 
Reform, House of Representatives: 

Why GAO Did This Study: 

The Department of Energy (DOE) is working to obtain a license from the 
Nuclear Regulatory Commission (NRC) to construct a nuclear waste 
repository at Yucca Mountain in Nevada. The project, which began in the 
1980s, has been beset by delays. In a 2004 report, GAO raised concerns 
that persistent quality assurance problems could further delay the 
project. Then, in 2005, DOE announced the discovery of employee e-mails 
suggesting quality assurance problems, including possible falsification 
of records. Quality assurance, which establishes requirements for work 
to be performed under controlled conditions that ensure quality, is 
critical to making sure the project meets standards for protecting 
public health and the environment. 

GAO was asked to examine (1) the history of the project’s quality 
assurance problems, (2) DOE’s tracking of these problems and efforts to 
address them since GAO’s 2004 report, and (3) challenges facing DOE as 
it continues to address quality assurance issues within the project. 

What GAO Found: 

DOE has had a long history of quality assurance problems at the Yucca 
Mountain project. In the 1980s and 1990s, DOE had problems assuring NRC 
that it had developed adequate plans and procedures related to quality 
assurance. More recently, as it prepares to submit a license 
application for the repository to NRC, DOE has been relying on costly 
and time-consuming rework to resolve lingering quality assurance 
problems uncovered during audits and after-the-fact evaluations. 

DOE announced, in 2004, that it was making a commitment to continuous 
quality assurance improvement and that its efforts would be tracked by 
performance indicators that would enable it to assess progress and 
direct management attention as needed. However, GAO found that the 
project’s performance indicators and other key management tools were 
not effective for this purpose. For example, the management tools did 
not target existing areas of concern and did not track progress in 
addressing them. The tools also had weaknesses in detecting and 
highlighting significant problems for management attention. 

DOE continues to face quality assurance and other challenges. First, 
DOE is engaged in extensive efforts to restore confidence in scientific 
documents because of the quality assurance problems suggested in the 
discovered e-mails between project employees, and it has about 14 
million more project e-mails to review. Second, DOE faces quality 
assurance challenges in resolving design control problems associated 
with its requirements management process—the process for ensuring that 
high-level plans and regulatory requirements are incorporated into 
specific engineering details. Problems with the process led to the 
December 2005 suspension of certain project work. Third, DOE continues 
to be challenged to manage a complex program and organization. 
Significant personnel and project changes initiated in October 2005 
create the potential for confusion over roles and responsibilities—a 
situation DOE found to contribute to quality assurance problems during 
an earlier transition. 

View of Yucca Mountain and the Exploratory Tunnel for the Repository: 

[See PDF for image] 

[End of figure] 

What GAO Recommends: 

GAO recommends five actions DOE can take to improve the project’s 
management tools and identify and address quality assurance and other 
problems. 

In oral comments, DOE agreed with GAO’s recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-06-313. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Jim Wells at (202) 512-
3841 or wellsj@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

DOE Has a Long History of Quality Assurance Problems at Yucca Mountain 
and Is Relying on Costly and Time-Consuming Measures to Correct 
Problems Before Submitting Its License Application for the Repository: 

DOE Cannot Be Certain Its Efforts to Improve Quality Assurance Have 
Been Effective Because of Weaknesses in Tracking Progress and 
Identifying Problems: 

DOE's 'New Path Forward' for Preparing to Submit Its License 
Application Faces Substantial Quality Assurance and Other Challenges: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Yucca Mountain Project Employee Concerns Programs: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Visibility of Management Improvement Initiatives' 
Effectiveness Indicators in Annunciator Panel When Last Prepared (Using 
August 2005 Data): 

Table 2: Key Indicators for Processes with Quality Elements, Their 
Intended Focus, and Number of Times They Changed (March through August 
2005): 

Table 3: Employee Concerns Opened for Investigation under DOE's 
Employee Concerns Program by Category of Concern, January through 
November 2005: 

Table 4: Employee Concerns Opened for Investigation under BSC's 
Employee Concerns Program by Category of Concern, January through 
November 2005: 

Figures: 

Figure 1: A Yucca Mountain Project Scientist Conducts Water 
Infiltration Tests inside Yucca Mountain: 

Figure 2: The 1.7-Mile Tunnel Built for Scientific Studies near the 
Potential Repository Area: 

Abbreviations: 

AMR: Analysis and Model Report: 

BSC: Bechtel/SAIC Company, LLC: 

CAP: Corrective Action Program: 

CR: condition report: 

DOE: Department of Energy: 

EPA: Environmental Protection Agency: 

Initiatives: Management Improvement Initiatives: 

INPO: Institute of Nuclear Power Operations: 

LSN: Licensing Support Network: 

NRC: Nuclear Regulatory Commission: 

OCRWM: Office of Civilian Radioactive Waste Management: 

RIT: Regulatory Integration Team: 

USGS: U.S. Geological Survey: 

Letter March 17, 2006: 

The Honorable Jon C. Porter:
Chairman: 
Subcommittee on the Federal Workforce and Agency Organization: 
Committee on Government Reform: 
House of Representatives: 

Dear Mr. Chairman: 

The nuclear waste created as a by-product of the nuclear power process 
in reactors can remain highly radioactive for hundreds of thousands of 
years, and will require proper disposal to protect public health and 
the environment. Over 50,000 metric tons of this waste is currently 
being stored at 72 sites around the country, principally at commercial 
nuclear power plants. These wastes have been accumulating for several 
decades in surface storage designed to be temporary. The Nuclear Waste 
Policy Act of 1982 obligated the Department of Energy (DOE) to 
construct an underground geological repository for permanent storage 
and begin accepting these wastes by January 31, 1998. However, it was 
not until 2002, after more than 15 years of scientific study, that 
Yucca Mountain in Nevada was approved by Congress as a suitable 
location for the repository. DOE is continuing to experience delays, 
and it does not currently have a schedule for when construction of the 
repository will begin. The project to build and operate a repository at 
Yucca Mountain is highly complex. It is also highly controversial among 
some of the public, in large part, because of their concern that the 
repository may not be adequate, over the long term, to prevent the 
release of radioactive material to the environment. DOE has established 
quality assurance procedures to ensure that its work relative to the 
project and the technical information it produces are of high quality 
and defensible. However, persistent problems with implementing these 
procedures and resulting questions about the quality of the work have 
significantly contributed to project delays. Resolving these quality 
issues is essential to proceeding with construction. 

To construct a repository at the Yucca Mountain site, DOE must obtain a 
license from the Nuclear Regulatory Commission (NRC). As part of the 
licensing process, DOE must demonstrate to NRC that its plans for the 
repository will meet standards for protecting public health and the 
environment from harmful exposure to the radioactive waste. The 
Environmental Protection Agency (EPA) set these standards in 2001, but 
as a result of a 2004 court ruling, EPA is proposing to revise the 
standards to extend the protection period from 10,000 years to 1 
million years.[Footnote 1] 

To demonstrate that it can meet these standards, DOE has been 
conducting scientific and technical studies at the Yucca Mountain site 
that will serve as supporting documentation for DOE's planned license 
application. For example, it has developed mathematical models to 
measure the probability that various combinations of natural and 
engineered (human-made) features of a repository could safely contain 
waste for the long term; the models take into account possible water 
infiltration through the mountain (see fig. 1), earthquakes, volcanic 
action, or other scenarios. Thus, one of DOE's most important tasks 
during the licensing process will be to demonstrate the adequacy of its 
data, software, and models. Accordingly, NRC requires nuclear 
facilities to develop a quality assurance program that ensures that the 
technical information submitted in support of a license application-- 
such as scientific data, models, and details on design and 
construction--is well documented and defensible. More specifically, 
data used to support conclusions about the safety and design of the 
repository must meet transparency and traceability standards. That is, 
the data must be clear in justifying and explaining any underlying 
assumptions, calculations, and conclusions, and must be capable of 
being traced back to original source materials. 

Figure 1: A Yucca Mountain Project Scientist Conducts Water 
Infiltration Tests inside Yucca Mountain: 

[See PDF for image] 

[End of figure] 

To meet NRC's requirements, DOE established a quality assurance program 
for the Yucca Mountain project. The program establishes requirements 
that scientific, design, engineering, and other work, such as 
procurement and record keeping, is to be performed under controlled 
conditions that ensure quality and enable the work to be verified by 
others. For example, the program establishes general requirements for 
calibrating equipment before conducting tests, stipulating when and how 
the equipment should be calibrated and how to document the results. The 
project's line organizations, which are responsible for carrying out 
various functions or aspects of the work, then create their own 
policies and procedures to implement the requirements. 

Project employees are required to follow such procedures to help ensure 
the reliability of project information. Quality assurance auditors 
periodically verify that the procedures have been followed. Project 
employees, including quality assurance auditors, are required to 
identify when procedures are not being followed or when they encounter 
problems with the procedures. These problems can be identified in 
"condition reports" under the project's Corrective Action Program, 
which establishes procedures for the prompt identification and 
correction of problems. Alternatively, project employees can submit 
problems for resolution through the Employee Concerns Program, which 
allows for submissions to be confidential or anonymous. 

Because quality assurance plays a key role in ensuring that the 
information DOE uses to support its license application is of high 
quality and fully defensible, problems in this area raise concerns 
about delays to DOE's submission and NRC's review of the license 
application. In April 2004, for example, we reported that recurring 
quality assurance problems at the Yucca Mountain project could delay 
the licensing and operation of the repository.[Footnote 2] As we noted, 
a 2004 NRC evaluation found quality assurance problems such as data 
that could not be readily traced back to their sources. NRC indicated 
that unless DOE rectified such problems before submitting the license 
application, NRC could be in the position of requesting large volumes 
of additional information, which could prevent it from making a 
decision on the license within the time required by law. Then, in early 
2005, DOE reported it had discovered a series of e-mail messages among 
some U.S. Geological Survey (USGS) employees working on the Yucca 
Mountain project under a contract with DOE that appeared to imply that 
workers had falsified records for scientific work. Several of these 
messages, written in the late 1990s, appeared to show disdain for the 
project's quality assurance program and its requirements. As a result 
of these e-mails, DOE is engaging in an extensive review of records to 
restore confidence in scientific documents that will be used to support 
its license application. 

DOE's recent efforts to better manage quality assurance problems 
include its Management Improvement Initiatives (Initiatives), which 
began in 2002 and were reported completed in April 2004. The 
Initiatives' purpose was to ensure that work and products consistently 
met quality objectives and were fully defensible by establishing a 
foundation for continuous improvement in areas of identified management 
weaknesses. In our 2004 report, we concluded that, while DOE considered 
the Initiatives to have been completed, it could not assess their 
effectiveness in addressing the management weaknesses because its 
performance goals lacked objective measures and time frames for 
determining success.[Footnote 3] By the end of the Initiatives, DOE had 
established two tools to alert management about quality-related and 
other problems: (1) a one-page summary of performance indicators for 
key project activities and processes (the summary, which DOE refers to 
as a "panel," is prepared monthly for discussion and action by project 
managers) and (2) quarterly trend evaluation reports analyzing patterns 
and trends in problems identified through the Corrective Action 
Program. Then, in October 2005, DOE initiated planning for an 
aggressive series of changes to the facility design, organization, and 
management of the Yucca Mountain project. This effort, known as the 
"new path forward," is intended to address quality assurance and other 
challenges prior to submission of a license application. According to 
the project's Acting Director, DOE will be considering changes in 
performance indicators and other management tools to better support the 
new path forward. 

In this context, you requested that we provide additional information 
on the project's quality assurance problems and DOE's efforts to 
correct them. As agreed with your office, this report discusses (1) the 
history of the project's quality assurance problems since its start in 
the 1980s, (2) DOE's tracking of quality problems and progress 
implementing quality assurance requirements since our April 2004 
report, and (3) challenges DOE faces as it continues to address quality 
assurance issues at the project. In addition, you asked for information 
about concerns raised in recent years through the project's Employee 
Concerns Program, which is provided in appendix II. 

To determine the history of quality assurance problems, we reviewed 
previous GAO, DOE, and NRC documents, visited the project, and 
interviewed officials from DOE, NRC, and Bechtel/SAIC Company, LLC 
(BSC), which is DOE's management contractor for the Yucca Mountain 
project. To assess DOE's tracking of quality-related problems and 
progress in addressing them, we examined management tools and 
associated documentation, such as monthly indicator panels and 
quarterly trend reports, and interviewed BSC and DOE officials 
regarding those tools. To identify current quality assurance and other 
challenges, we attended quarterly NRC management meetings, interviewed 
the Acting Director and other senior managers of the DOE project, and 
gathered information on management turnover. Due to the criminal 
investigation under way related to possible falsification of records 
implied in USGS e-mail exchanges, we did not examine the investigated 
issues beyond confirming that a concern about the e-mails had been 
submitted to the Employee Concerns Program. However, to determine if 
concerns about other instances of potential falsification of records 
had been raised by project employees, we reviewed employee concerns 
filed with the project's Employee Concerns Program from January 2004 to 
December 2005. More information on our scope and methodology is 
provided in appendix I. We conducted our work from July 2005 through 
January 2006 in accordance with generally accepted government auditing 
standards. 

Results in Brief: 

DOE has had a long history of quality assurance problems at the Yucca 
Mountain project. In the late 1980s and early 1990s, DOE had problems 
assuring NRC that it had developed adequate plans and procedures 
related to quality assurance. For example, as GAO reported in 1988, NRC 
had found that DOE's quality assurance procedures were inadequate and 
its efforts to independently identify and resolve weaknesses in the 
procedures were ineffective. By the late 1990s, DOE had largely 
addressed NRC's concerns about its plans and procedures, but its own 
audits identified quality assurance problems with the data, software, 
and models used in the scientific work supporting its potential license 
application. For example, in 1998, a team of project personnel 
determined that 87 percent of the models used to simulate the site's 
natural and environmental conditions, and to demonstrate the future 
repository's performance over time, did not comply with requirements 
for demonstrating their accuracy in predicting geologic events. More 
recently, as it prepares to submit the license application for the 
planned repository to NRC, DOE has been relying on costly and time- 
consuming rework to resolve lingering quality assurance concerns. For 
example, to address problems with the transparency and traceability of 
scientific work in technical documents, DOE implemented, in the spring 
of 2004, a roughly $20 million, 8-month project called the Regulatory 
Integration Team. This effort involved about 150 full-time employees 
from DOE, USGS, and multiple national laboratories, such as Sandia, Los 
Alamos, and Lawrence Livermore, working to inspect technical documents 
to identify and resolve quality problems. 

DOE cannot be certain that its efforts to improve the implementation of 
its quality assurance requirements have been effective because it 
adopted management tools that did not target existing management 
concerns and did not track progress with significant and recurring 
problems. Although DOE announced, in 2004, that it was making a 
commitment to continuous quality assurance improvement and that its 
efforts would be tracked by performance indicators that would enable it 
to assess progress and direct management attention as needed, its 
adopted management tools have not been effective for this purpose. 
Specifically, the one-page summary, or "panel," of selected performance 
indicators that project managers used in monthly management meetings 
was not an effective tool for assessing progress. The indicators 
selected for the panel poorly represented the major management concerns 
and changed frequently. For example, the panel did not include an 
indicator to represent the management concern about unclear roles and 
responsibilities--a problem that could undermine accountability within 
the project. Use of the indicator panel was discontinued in late 2005, 
and DOE is deciding on a tool to replace it. Moreover, a second 
management tool--trend evaluation reports--also did not track relevant 
concerns. The reports generally had technical weaknesses for 
identifying recurrent and significant problems and inconsistently 
tracked progress in resolving the problems. For example, lacking 
reliable data and an appropriate performance benchmark for determining 
the significance of human errors as a cause of quality problems, DOE's 
trend reports offered no clear basis for tracking progress on such 
problems. In addition, under the trend reports' rating categories, the 
rating assigned to convey the significance of a problem was overly 
influenced by a judgment that there were already ongoing management 
actions to address the problem, rather than solely assessing the 
problem's significance. For example, the trend report's rating of one 
particular problem at the lowest level of significance did not 
accurately describe the problem or sufficiently draw management's 
attention to it. 

Before DOE submits a license application, its aggressive "new path 
forward" effort faces substantial quality assurance and other 
challenges. First, the March 2005 announcement of the discovery of USGS 
e-mails suggesting the possible falsification of quality assurance 
records has resulted in extensive efforts to restore confidence in 
scientific documents, and DOE is conducting a wide-ranging review of 
approximately 14 million e-mails to determine whether they raise 
additional quality assurance issues. Such a review creates a challenge 
not just because of the sheer volume of e-mails to be reviewed, but 
also because DOE will have to decipher their meaning and determine 
their significance, sometimes without clarification from authors who 
have left the project. Furthermore, if any of the e-mails raise quality 
assurance concerns, further review, inspection, or rework may need to 
be performed to resolve any newfound problems. Second, DOE faces 
quality assurance challenges in resolving design control problems 
associated with an inadequate requirements management process--the 
process responsible for ensuring that broad plans and regulatory 
requirements affecting the project are tracked and incorporated into 
specific engineering details. In December 2005, DOE issued a stop-work 
order on some design and engineering work until DOE can determine that 
the requirements management process has been improved. Third, DOE 
continues to be challenged by managing a changing and complex program 
and organization. Significant project changes initiated in October 2005 
under the new path forward create the potential for confusion over 
accountability as roles and responsibilities change--a situation DOE 
found to contribute to quality assurance problems during an earlier 
transition period. For example, a proposed reorganization, establishing 
a lead laboratory to assist the project, not only would have to be 
effectively managed, but also would introduce a new player whose 
accountability DOE would have to ensure. DOE has also experienced 
turnover in 9 of 17 key management positions since 2001--including 
positions related to quality assurance--that has created management 
continuity challenges. For example, the director position for the 
project has been occupied by three individuals since 1999 and is 
currently occupied by an acting director. Since DOE is still 
formulating its plans, it is too early to determine whether its new 
effort will effectively resolve these challenges. 

We are making recommendations to DOE aimed at improving the 
effectiveness of its management tools for monitoring performance in key 
areas, including quality assurance, by improving the tools' ability to 
identify problems and track progress in addressing them. We provided 
DOE and NRC with draft copies of this report for their review and 
comment. In comments, DOE agreed with our recommendations. Both DOE and 
NRC provided technical and editorial comments that we incorporated into 
the report, as appropriate. 

Background: 

Congress enacted the Nuclear Waste Policy Act of 1982 to establish a 
comprehensive policy and program for the safe, permanent disposal of 
commercial spent fuel and other highly radioactive wastes in one or 
more mined geologic repositories. The act charged DOE with (1) 
establishing criteria for recommending sites for repositories; (2) 
"characterizing" (investigating) three sites to determine each site's 
suitability for a repository (1987 amendments to the act directed DOE 
to investigate only the Yucca Mountain site); (3) recommending one 
suitable site to the President, who, if he considered the site 
qualified for a license application, would submit a recommendation to 
Congress; and (4) seeking a license from NRC to construct and operate a 
repository at the approved site. The act created the Office of Civilian 
Radioactive Waste Management within DOE to manage its nuclear waste 
program. 

Since the 1980s, DOE has spent years conducting site characterization 
studies at the Yucca Mountain site to determine whether it is suitable 
for a high-level radioactive waste and spent nuclear fuel repository. 
DOE, for example, has completed numerous scientific studies of the 
mountain and its surrounding region for water flow and the potential 
for rock movement, including volcanoes and earthquakes that might 
adversely affect the performance of the repository. To allow scientists 
and engineers greater access to the rock being studied, DOE excavated 
two tunnels for studying the deep underground environment: (1) a five- 
mile main tunnel that loops through the mountain, with several research 
areas or alcoves connected to it; and (2) a 1.7-mile tunnel that 
crosses the mountain (see fig. 2). This second tunnel allows scientists 
to study properties of the rock and the behavior of water near the 
potential repository area. In July 2002, Congress approved the 
President's recommendation of the Yucca Mountain site for the 
development of a repository. 

Figure 2: The 1.7-Mile Tunnel Built for Scientific Studies near the 
Potential Repository Area: 

[See PDF for image] 

[End of figure] 

The Yucca Mountain project is currently focused on preparing an 
application to obtain a license from NRC to construct a repository. The 
required application information includes both repository design work 
and scientific analyses. DOE is engaged in necessary tasks such as 
compiling information and writing sections of the license application, 
and is conducting technical exchanges with NRC staff and addressing key 
technical issues identified by NRC to ensure that sufficient supporting 
information is provided. It also plans to further develop the design of 
the repository, including revised designs for the repository's surface 
facilities and canisters to hold the waste. DOE is also identifying and 
preparing potentially relevant documentary material that it is required 
to make available on NRC's Web-based information system, known as the 
Licensing Support Network. This is a critical step because DOE is 
required to certify that the documentary material has been identified 
and made electronically available no later than 6 months in advance of 
submitting the license application.[Footnote 4] 

In February 2005, DOE announced that it does not expect the repository 
to open until 2012 at the earliest, which is more than 14 years later 
than the 1998 goal specified by the Nuclear Waste Policy Act of 1982. 
More recently, the conference report for DOE's fiscal year 2006 
appropriations observed that further significant schedule slippages for 
submitting a license application are likely. Further delays could arise 
from factors such as the time needed for EPA to establish revised 
radiation standards for Yucca Mountain and for DOE to revise its 
technical documents in response. Such delays could be costly because 
nuclear utilities, which pay for most of the disposal program through a 
fee on nuclear power, have sued DOE, seeking damages for not starting 
the removal of spent nuclear fuel from storage at commercial reactors 
by the 1998 deadline. Estimates of the potential damages vary widely, 
from DOE's estimate of about $5 billion to a nuclear industry's 
estimate of about $50 billion, but the cost for the damages will likely 
rise if there are further delays to opening the repository. 

Given these schedule slippages, Congress has considered other options 
for managing existing and future nuclear wastes, such as centralized 
interim storage at one or more DOE sites. The conference report for 
DOE's fiscal year 2006 appropriations directed DOE to develop a spent 
nuclear fuel recycling plan to reuse the fuel. However, according to 
the policy organization of the nuclear energy industry, no 
technological option contemplated will eliminate the need to ultimately 
dispose of nuclear waste in a geologic repository. 

In October 2005, the project's Acting Director issued a memorandum 
calling for the development of wide-ranging plans for the "new path 
forward," DOE's effort to address quality assurance and other 
challenges prior to applying for a license. To restore confidence in 
scientific documents that will support the license application, some of 
the plans will address the need to review and replace USGS work 
products, a requirement for USGS to certify its scientific work 
products, and establishing a lead national laboratory to assist the 
project. Other plans are focused on a new simplified design for the 
waste canisters and repository facilities, a design that is expected to 
improve the safety and operation of the repository by eliminating the 
need to directly handle and process the spent fuel at the repository. 
Further, this aggressive effort called for management changes, 
including a transition plan; more rigorous project management, 
including a new baseline schedule; rescoping existing contracts and 
developing new contracts; tracking project hiring actions; a financial 
plan; and new reporting indicators. 

After DOE submits the license application, NRC plans to take 90 days to 
examine the application for completeness to determine whether DOE has 
addressed all NRC requirements. One of the reviews for completeness 
will include an examination of DOE's documentation of the quality 
assurance program to assess whether it addresses all NRC criteria. 
These criteria include, among other things, organization, design 
control, document control, corrective actions, quality assurance 
records, and quality audits. If it deems any part of the application is 
incomplete, NRC may either reject the application or require that DOE 
furnish the necessary documentation before proceeding with the detailed 
technical review of the application. If it deems the application is 
complete, NRC will docket the application, indicating its readiness for 
a detailed technical review.[Footnote 5] 

Once the application is accepted and placed on the docket, NRC will 
conduct its 18-month technical review of the application to determine 
if the application meets all NRC requirements, including the soundness 
of scientific analyses and preliminary facility design, and NRC quality 
assurance criteria. If NRC discovers problems with the technical 
information used to support the application, it may conduct specific 
reviews, including inspections, to determine the extent and effect of 
the problem. Because the data, models, and software used in modeling 
repository performance are integral parts of this technical review, 
quality assurance plays a key role since it is the mechanism used to 
verify the accuracy of the information DOE presents in the application. 
NRC may conduct reviews, including inspections, of the quality 
assurance program if technical problems are identified that are 
attributable to quality problems. NRC will hold public hearings chaired 
by its Atomic Safety and Licensing Board to examine specific topics. 
After completing the proceedings, the board will forward its initial 
decision to the NRC commissioners for their review. Finally, within 3 
to 4 years from the date that NRC dockets the application, NRC will 
make a decision to grant the construction authorization, reject the 
application, or grant the construction authorization with 
conditions.[Footnote 6] NRC will grant a construction authorization 
only if it concludes from its reviews that the repository would meet 
its reasonable expectation that the safety and health of workers and 
the public would be protected. 

DOE Has a Long History of Quality Assurance Problems at Yucca Mountain 
and Is Relying on Costly and Time-Consuming Measures to Correct 
Problems Before Submitting Its License Application for the Repository: 

DOE has repeatedly experienced quality assurance problems with its work 
on the Yucca Mountain project. In the late 1980s, DOE had been 
challenged to fix and develop adequate plans and procedures related to 
quality assurance. By the late 1990s, audits by GAO, DOE, and others 
identified recurring quality assurance problems with several aspects of 
key scientific data, models, and software. Currently, in preparing to 
submit the license application to NRC, DOE is relying on costly and 
time-consuming rework to resolve lingering quality assurance problems 
with the transparency and traceability of data and in project design 
and engineering documents uncovered during audits and after-the-fact 
evaluations. 

DOE Has Had Problems Implementing and Maintaining an Effective Quality 
Assurance Program: 

DOE has a long-standing history of attempting to address NRC concerns 
about its quality assurance program. Although NRC will have 
responsibility for regulating the construction, operation, and 
decommissioning (closure) phases of the project, its regulatory and 
oversight role does not begin until DOE submits a license application. 
As a result, NRC's role in the project has been limited to providing 
guidance to DOE to ensure an understanding of NRC regulations and that 
the years of scientific and technical work will not later be found 
inadequate for licensing purposes. Specifically, since 1984, NRC has 
agreed to point out problems it identifies with the quality assurance 
program so that DOE can take timely corrective action. Initially, this 
NRC guidance was mainly focused on ensuring that DOE had the necessary 
quality assurance organization, plans, and procedures. 

As we reported in 1988, NRC had reviewed DOE's quality assurance plans 
and procedures comprising the principal framework of its quality 
assurance program, and concluded that they were inadequate and did not 
meet NRC requirements.[Footnote 7] NRC also concluded that DOE's 
efforts to independently identify and resolve weaknesses in the plans 
and procedures were ineffective. After observing DOE quality assurance 
audits, NRC determined that the audits were ineffective for measuring 
whether quality assurance procedures were being effectively 
implemented. Further, NRC identified additional concerns, during the 
1980s, related to DOE management and organizational deficiencies 
relating to the quality assurance program. Specifically, among other 
things, NRC found the following: 

* DOE had a small staff and relied heavily on contractors to provide 
quality assurance oversight. Based on its experience in regulating 
nuclear power plants, NRC found that these types of organizations 
frequently developed major quality-related problems. 

* DOE had indirect project control, with administrative and functional 
control over the project split between different offices. NRC found 
that such project control arrangements tend to have serious quality 
assurance-related problems because conflicts can arise between quality 
and other organizational goals, such as cost and schedule. 

* During a 1984 NRC visit to Nevada, DOE project participants had 
expressed the opinion that quality assurance is "unnecessary, 
burdensome, and an imposition." Further, in 1986, DOE issued a stop- 
work order to the USGS based on a determination that USGS staff did not 
appreciate the importance of quality assurance and that USGS work would 
not meet NRC expectations. NRC believed that organizational attitudes 
can indicate whether a project is likely to experience problems 
relating to quality assurance and found such examples troublesome. 

Finally, based in part on the information obtained from its oversight 
activities, NRC concluded, in 1989, that DOE and its key contractors 
had yet to develop and implement an acceptable quality assurance 
program. 

However, by March 1992, NRC came to the conclusion that DOE had made 
significant progress in improving its quality assurance program. NRC 
noted that DOE had addressed many of its concerns, specifically that, 
among other things, (1) all of the contractor organizations had 
developed and were in the process of implementing quality assurance 
programs that met NRC requirements, (2) quality assurance management 
positions had been filled with full-time DOE personnel with appropriate 
knowledge and experience, and (3) DOE had demonstrated that it is 
capable of evaluating and correcting deficiencies in the overall 
quality assurance program. Nevertheless, in October 1994, NRC found 
problems with quality assurance, particularly with the site 
contractor's ability to effectively implement corrective actions and 
DOE's ability to oversee the site contractor's quality assurance 
program. 

Recurring Issues with Project Data, Models, and Software Illustrate 
DOE's Difficulties Addressing Quality Assurance Problems: 

As DOE's quality assurance program matured, it resolved NRC concerns 
about its organization, plans, and procedures, and in the late 1990s 
began successfully detecting new quality assurance problems in three 
areas critical to the repository's successful performance: the adequacy 
of the data sources, the validity of scientific models, and the 
reliability of computer software developed at the site. These problems 
surfaced in 1998 when DOE began to run the initial version of its 
performance assessment model. Specifically, DOE was unable to ensure 
that critical project data had been collected and tracked back to the 
original sources. In addition, DOE did not have a standardized process 
for developing scientific models used to simulate a variety of geologic 
events or an effective process for ensuring that computer software used 
to support the scientific models would work properly. As required by 
DOE's quality assurance procedures, the department conducted a root 
cause analysis and issued a corrective action plan in 1999. After 
corrective actions were taken, DOE considered the issues resolved. 

However, in 2001, similar deficiencies associated with models and 
software resurfaced. DOE attributed the recurrence to ineffective 
procedures and corrective actions, improper implementation of quality 
procedures by line managers, and personnel who feared reprisal for 
expressing quality concerns. Recognizing the need to correct these 
recurring problems, DOE conducted a comprehensive root cause analysis 
that included reviews of numerous past self-assessments and independent 
program assessments, and identified weaknesses in management systems, 
quality processes, and organization roles and responsibilities. 
Following the analysis, in July 2002, DOE issued its Management 
Improvement Initiatives (Initiatives) that addressed quality problems 
with software and models. In addition, DOE added other corrective 
actions to address management weaknesses that it found in areas such as 
roles and responsibilities, quality assurance processes, written 
procedures, corrective action plans, and work environment. 

However, DOE continued to face difficulties in resolving quality 
assurance problems concerning the data, software, and modeling to be 
used in support of the licensing application: 

* Data management. As part of NRC's quality assurance requirements, 
data used to support conclusions about the safety and design of the 
repository must be either collected under a quality assurance program 
or subjected to prescribed testing procedures to ensure the data are 
accurate for their intended use. In addition, the data supporting these 
conclusions must also be traceable back to its original source. In 
1998, DOE identified quality assurance problems with the quality and 
traceability of data--specifically that some data had not been properly 
collected or tested to ensure their accuracy and that data used to 
support scientific analysis could not be properly traced back to their 
source. DOE again found similar problems in April and September 2003, 
when a DOE audit revealed that some data sets did not have the 
documentation necessary to trace them back to their sources; the 
processes for data control and management were unsatisfactory; and 
faulty definitions were developed, which allowed unqualified data to be 
used. 

* Software management. DOE quality assurance procedures require that 
software used to support analysis and conclusions about the performance 
and safety of the repository be tested or created in such a way to 
ensure that it is reliable. From 1998 to 2003, multiple DOE audits 
found recurring quality assurance problems that could affect confidence 
in the adequacy of software codes. For example, in 2003, DOE auditors 
found problems related to software similar to those found previously in 
areas such as technical reviews, software classification, planning, 
design, and testing. Further, a team of industry professionals hired by 
DOE to assess quality assurance problems with software reported in 
February 2004 that these problems kept recurring because DOE did not 
assess the effectiveness of its corrective actions and did not 
adequately identify the root causes of the problems. 

* Model validation. Models are used to simulate natural and 
environmental conditions at Yucca Mountain, and to demonstrate the 
performance of the future repository over time. However, before models 
can be used to support the license application, DOE must demonstrate 
through a process called validation that the models are able to 
accurately predict geologic events. In 1998, a team of project 
personnel evaluated the models and determined that 87 percent did not 
comply with the validation requirements. In 2001, and again in 2003, 
DOE audits found that project personnel were not properly following 
procedures--specifically in the areas of model documentation, model 
validation, and checking and review. Further, the 2003 audit concluded 
that previous corrective actions designed to improve validation and 
reduce errors in model reports were not fully implemented. 

After many years of working to address these quality assurance problems 
with data, software, and models, DOE had mostly resolved these problems 
and closed the last of the associated condition reports by February 
2005. 

DOE Is Now Relying on Costly and Time-Consuming Rework to Resolve 
Additional Problems: 

As DOE prepares to submit the Yucca Mountain project license 
application to NRC, it has relied on costly and time consuming rework 
to ensure that the documents supporting the application are accurate 
and complete. Specifically, DOE has relied on inspections and rework by 
DOE personnel to resolve quality assurance problems with the 
traceability and transparency of technical work products. These efforts 
to deal with quality problems at the end, rather than effectively 
ensuring that work organizations are producing quality products from 
the beginning, add to the project's cost and could potentially delay 
DOE's submission of the license application to NRC. In addition, DOE's 
efforts indicate that some corrective actions have been ineffective in 
resolving problems with the quality assurance process. Further, DOE is 
now detecting quality assurance problems in design and engineering work 
that are similar to the quality assurance problems it experienced with 
its scientific work in the late 1990s. 

Although DOE did not initiate its major effort to address these 
problems until 2004, the department and NRC for years had known of 
quality assurance problems with the traceability and transparency of 
technical work products called Analysis and Model Reports (AMR). AMRs 
are a key component of the license application, and contain the 
scientific analysis and modeling data demonstrating the safety and 
performance of the planned repository. Among other quality 
requirements, AMRs must be traceable back to their original source 
material and data, and must also be transparent in justifying and 
explaining their underlying assumptions, calculations, and conclusions. 
In 2003, based in part on these problems as well as DOE's long-standing 
problems with data, software, and modeling, NRC conducted an 
independent evaluation of three AMRs. The scope of the review was to 
determine if the AMRs met NRC requirements for being traceable, 
transparent, and technically appropriate for their use in the license 
application. NRC found significant problems.[Footnote 8] First, in some 
cases DOE was not transparent in explaining the basis on which it was 
reaching conclusions. For example, in some circumstances, DOE selected 
a single value from a range of data without sufficient justification. 
Other times, DOE did not explain how a range of experimental conditions 
were representative of repository conditions. Second, where DOE did 
sufficiently explain the basis for a conclusion, it did not always 
provide the necessary technical information, such as experimental data, 
analysis, or expert judgment, to trace the support for that explanation 
back to source materials. For example, DOE did not explain how 
information on one type of material provided an appropriate comparison 
for another material. Moreover, while DOE had identified similar 
problems in the past, the actions taken to correct them did not 
identify and resolve other deficiencies. NRC concluded that these 
findings suggested that other AMRs possibly had similar problems, and 
that if not resolved, such problems could delay NRC's review of the 
license application as it would need to conduct special inspections to 
resolve any issues it found with the quality of technical information. 

To address problems of traceability and transparency, DOE in the spring 
of 2004 initiated an effort called the Regulatory Integration Team 
(RIT) to perform a comprehensive inspection and rework of the AMRs to 
ensure they met NRC requirements and expectations.[Footnote 9] 
According to DOE officials, the RIT involved roughly 150 full-time 
personnel from DOE, USGS, and multiple national laboratories such as 
Sandia, Los Alamos, and Lawrence Livermore. First, the RIT screened all 
of the approximately 110 AMRs and prioritized its efforts on 89 that 
needed additional rework. Ten AMRs were determined to be acceptable, 
and 11 were canceled because they were no longer needed to support the 
license application. According to DOE officials, approximately 8 months 
later, the RIT project was completed at a cost of about $20 million, 
with a total of over 3,700 problems and issues addressed or corrected. 
In February 2005, in a letter to DOE, the site contractor stated that 
the RIT effort was successful and that the AMRs had been revised to 
improve traceability and transparency. 

Subsequently, however, additional problems with traceability and 
transparency have been identified, requiring further inspections and 
rework. For example, after the March 2005 discovery of e-mails from 
USGS employees written between May 1998 and March 2000 implying that 
employees had falsified documentation of their work to avoid quality 
assurance standards, DOE initiated a review of additional AMRs that 
were not included in the scope of the 2004 RIT review. The additional 
AMRs contained scientific work performed by the USGS employees and had 
been assumed by the RIT to meet NRC requirements for traceability and 
transparency. However, according to DOE officials, DOE's review 
determined that these AMRs did not meet NRC's standards, and additional 
rework was required. Further, similar problems were identified as the 
focus of the project shifted to the design and engineering work 
required for the license application. In February 2005, the site 
contractor determined that in addition to problems with AMRs, similar 
traceability and transparency problems existed in the design and 
engineering documents that comprise the Safety Analysis Report--the 
report necessary for demonstrating to NRC how the facilities and other 
components of the repository site will meet the project's health, 
safety, and environmental goals and objectives. In a root cause 
analysis of this problem, the site contractor noted that additional 
resources were needed to inspect and rework the documents to correct 
the problems. 

DOE Cannot Be Certain Its Efforts to Improve Quality Assurance Have 
Been Effective Because of Weaknesses in Tracking Progress and 
Identifying Problems: 

DOE cannot be certain that it has met continuous improvement goals for 
implementing its quality assurance requirements, a commitment DOE made 
at the closure of its Management Improvement Initiatives (Initiatives) 
in April 2004. At that time, DOE told us it expected that the progress 
achieved with the initiatives would continue and that its performance 
indicators would enable it to assess further progress and direct 
management attention as needed. However, DOE's performance indicators, 
as well as a second management tool--trend evaluation reports--have not 
been effective for this purpose. More specifically, the indicators 
panel did not highlight the areas of concern covered by the initiatives 
and had weaknesses in assessing progress because the indicators kept 
changing. The trend evaluation reports also did not focus on tracking 
the concerns covered by the Initiatives, had technical weaknesses for 
identifying significant and recurring problems, had inconsistently 
tracked progress in addressing problems, and could not fully analyze 
projectwide problems.[Footnote 10] In addition, the trend reports' 
tracking of problems for which corrective actions were already being 
taken was at times overly influenced by judgments about whether 
additional management action was warranted rather than the problems' 
significance. 

The Panel's Focus and Frequent Changes Hindered the Tracking of 
Progress with Management Concerns and Quality Problems: 

By the time that the actions called for by the Initiatives had been 
completed in April 2004, project management had already developed the 
indicators panel, which DOE refers to as the annunciator panel, to use 
at monthly management meetings to monitor project performance. The 
panel was a single page composed of colored blocks representing 
selected performance indicators and their rating or level of 
performance. A manager viewing the panel would be able to quickly see 
the color rating of each block or indicator. For example, red indicated 
degraded or adverse performance warranting significant management 
attention; yellow indicated performance warranting increased management 
attention or acceptable performance that could change for the worse; 
and green indicated good performance. The panel represented a hierarchy 
of indicators in which the highest level, or primary, indicators were 
shown; secondary indicators that determined the primary indicators' 
ratings were shown for some primary indicators; but lower third-or 
fourth-level indicators were not shown. Our review analyzed a subset of 
these indicators that DOE designated as the indicators that best 
predict performance in areas affecting quality. While we were 
conducting our review, DOE suspended preparation of the panel after 
August 2005 while it reconsiders its use of indicators to monitor 
project performance. DOE had also suspended preparation of the panel 
from late 2004 to early 2005 in order to make substantial revisions. 
These revisions were made, in part, to emphasize fewer, more important 
indicators for management attention. 

The Initiatives raised concerns about five key areas of management 
weakness as adversely affecting the implementation of quality assurance 
requirements: 

1. Roles and responsibilities were becoming confused as the project 
transitioned from scientific studies to activities supporting 
licensing. The confusion over roles and responsibilities was 
undermining managers' accountability for results. The Initiatives' 
objective was to realign DOE's project organization to give a single 
point of responsibility for project functions, such as quality 
assurance and the Corrective Action Program, and hold the project 
contractor more accountable for performing the necessary work in 
accordance with quality, schedule, and cost requirements. 

2. Product quality was sometimes being achieved through inspections by 
the project's Office of Quality Assurance rather than being routinely 
implemented by the project's work organizations. As a result, the 
Initiatives sought to increase work organizations' responsibility for 
being the principle means for achieving quality. 

3. Work procedures were typically too burdensome and inefficient, which 
impeded work. The Initiatives sought to provide new user-friendly and 
effective procedures, when necessary, to allow routine compliance with 
safety and quality requirements. 

4. Multiple corrective action programs existed, processes were 
burdensome and did not yield useful management reports, and corrective 
actions were not completed in a timely manner. The Initiatives sought 
to implement a single program to ensure that problems were identified, 
prioritized, and documented and that timely and effective corrective 
actions were taken to preclude recurrence of problems. 

5. The importance of a safety-conscious work environment that fosters 
open communication about concerns was not understood by all managers 
and staff, and they had not been held accountable when inappropriately 
overemphasizing the work schedule, inadequately attending to work 
quality, and acting inconsistently in practicing the desired openness 
about concerns. Through issuing a work environment policy, providing 
training on the policy, and improving the Employee Concerns Program, 
the Initiatives sought to create an environment in which employees felt 
free to raise concerns without fear of reprisal and with confidence 
that issues would be addressed promptly and appropriately. 

As shown in table 1, the Initiatives' effectiveness indicators for 
tracking progress in addressing these management weaknesses did not 
have equivalent performance indicators visible in the annunciator panel 
when it was prepared for the last time, using August 2005 data. 

Table 1: Visibility of Management Improvement Initiatives' 
Effectiveness Indicators in Annunciator Panel When Last Prepared (Using 
August 2005 Data): 

Key area of management weakness identified in the Initiatives: Roles, 
responsibilities, accountability, authority; 
Effectiveness indicators from the Initiatives: An improving trend in 
quality and work schedule performance; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No integrated analysis of trends in quality and 
schedule performance; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel; Some indicators measured aspects of quality or 
schedule, but provided no integrated analysis of these trends. 

Effectiveness indicators from the Initiatives: A consistently 
decreasing trend in quality problems related to roles and 
responsibilities; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No aspect measured; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel. 

Key area of management weakness identified in the Initiatives: Quality 
assurance programs and processes; 
Effectiveness indicators from the Initiatives: The numbers of high-
priority (significant) quality problems that are self-identified are at 
least 80 percent of all significant quality problems; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: One indicator looked at work organizations' 
identification of problems, including less significant ones; 
GAO comments and observations: No indicator, focused only on 
significant problems, was visible in the panel; One fourth-level 
indicator tracked work organizations' identification of significant 
problems. 

Effectiveness indicators from the Initiatives: A decreasing trend in 
average time to resolve significant quality; problems and in number of 
delinquent corrective actions for significant quality problems; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: A new timeliness measure has been 
developed.[A]; 
GAO comments and observations: No indicator was visible in panel; 
Aspect of fourth-level indicator tracked average time of resolution. 

Key area of management weakness identified in the Initiatives: Work 
procedures; 
Effectiveness indicators from the Initiatives: A decreasing number of 
quality problems related to ineffective; Procedures; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No aspect measured; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel. 

Effectiveness indicators from the Initiatives: A decreasing trend in 
time needed to revise procedures; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No aspect measured; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel. 

Effectiveness indicators from the Initiatives: A decreasing trend in 
average time of interim procedure changes; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No aspect measured; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel. 

Key area of management weakness identified in the Initiatives: 
Corrective Action Program; 
Effectiveness indicators from the Initiatives: A decreasing trend in 
number of repetitive quality problems; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: No aspect measured; 
GAO comments and observations: No indicator was visible in, or 
underlies, the panel. 

Effectiveness indicators from the Initiatives: A decreasing trend in 
average time to resolve significant quality; problems; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: A new timeliness measure has been 
developed.[A]; 
GAO comments and observations: No indicator was visible in panel; 
Aspect of fourth-level indicator tracked average time of resolution. 

Effectiveness indicators from the Initiatives: Less than 10 percent of 
quality problems are resolved late; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: A new timeliness measure has been 
developed.[A]; 
GAO comments and observations: No indicator was visible in the panel; A 
third-level indicator tracked percentage of problems with timely 
resolution. 

Key area of management weakness identified in the Initiatives: Work 
environment; 
Effectiveness indicators from the Initiatives: A decreasing number of 
substantiated employee concerns for harassment, retaliation, 
intimidation, and discrimination; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: Aspects of this issue are measured by work 
environment indicators; 
GAO comments and observations: No indicator was visible in panel; A 
third-level indicator measured this performance. 

Effectiveness indicators from the Initiatives: Evaluation of routine 
employee concerns in less than 30 days, or; 90 days for complex 
employee concerns involving harassment or intimidation; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: Goals have remained at 30 and 90 days; 
GAO comments and observations: No indicator was visible in panel; Third-
level indicators measured the timely completion of routine and other 
concerns. 

Effectiveness indicators from the Initiatives: External evaluation of 
work environment shows positive changes; 
DOE response on coverage of the management weakness in the panel's 
performance indicators: External evaluation is accomplished through 
independent employee surveys, reflected in third-level indicators; 
GAO comments and observations: No indicator was visible in panel; Four 
third-level indicators were based on the employee surveys. 

Source: GAO analysis of DOE data. 

[A] New timeliness indicator was not implemented by the time of the 
final panel using August 2005 data. 

[End of table] 

Two of the Initiatives' key areas of concern--(1) roles, 
responsibilities, authority, and accountability; and (2) work 
procedures--and their associated effectiveness indicators were not 
represented in the panel's visible or underlying indicators. The 
Initiatives' effectiveness indicator for tracking trends in recurring 
problems also was not represented. In other cases, the Initiatives' 
effectiveness indicators were represented in underlying lower-level 
indicators that had very little impact on the rating of the visible 
indicator. An example is the Initiatives' indicator for timely 
completion of employee concerns. The panel's related visible indicator 
was work environment, whose rating was based on 4 secondary and 23 
tertiary indicators. Of the third-level indicators, two were for 
timeliness of completion of employee concerns, and combined they 
contributed 3 percent toward the rating of the work environment 
indicator. As a result of the weighting of these many underlying 
indicators, ratings for individual lower-level indicators could be 
different from the visible indicator. For example, in August 2005, the 
work environment indicator showed good performance. However, the 
ratings of four underlying indicators from the project's employee 
survey on the work environment--collectively accounting for 25 percent 
of the work environment indicator's score--indicated the need for 
increased management attention. Moreover, some of the Initiatives' 
indicators, such as the work organizations' self-identification of 
significant problems, had their impact on visible indicators diluted by 
the inclusion of other indicators that were not focused solely on the 
detection of significant problems. 

Another shortcoming of the annunciator panel was that frequent changes 
to the indicators hindered the ability to identify problems for 
management attention and track progress in resolving them. The 
indicators could change in many ways, such as changes in their 
definition, calculation, or data sources used in calculations, or from 
the deletion or addition of a subindicator. When such changes were made 
to the indicators, progress became less clear because changes in 
reported performance levels may have been the result of the indicator 
changes rather than actual performance changes. Some of the indicators 
for key project processes with quality elements changed from one to 
five times during the 8-month period from April 2004 through November 
2004. Even after the major revision of the panel in early 2005, most of 
the performance indicators tracking quality issues continued to change 
over the next 6 months--that is, from March 2005 through August 2005. 
As shown in table 2, only one of the five relevant indicators did not 
change during this period. One indicator was changed four times during 
the 6-month period, resulting in it being different in more months than 
it remained the same. 

Table 2: Key Indicators for Processes with Quality Elements, Their 
Intended Focus, and Number of Times They Changed (March through August 
2005): 

Indicators: Performance improvement; 
Intended focus: Effectiveness of self-assessment of quality and other 
issues, lessons learned, and Corrective Action Program; 
Number of months changed: 4. 

Indicators: Work management; 
Intended focus: Quality of work products and documents; 
Number of months changed: 1. 

Indicators: Safety-conscious work environment; 
Intended focus: Worker confidence in management support for raising 
quality and other concerns without fear of retaliation; management 
effectiveness in detecting and preventing retaliation for raising 
concerns; effectiveness of normal and alternative problem resolution; 
Number of months changed: 0. 

Indicators: Human performance; 
Intended focus: Preventing, detecting, and correcting human errors; 
Number of months changed: 3. 

Indicators: Quality performance; 
Intended focus: Composite of quality indicators, in areas of 
engineering products, self-assessment, Corrective Action Program, and 
work products and documents; 
Number of months changed: 1. 

Source: GAO analysis of DOE data. 

[End of table] 

Moreover, the panel was not always available to identify problems and 
track progress. The panel was not created for December 2004, January 
2005, and February 2005 because it was undergoing a major revision. At 
that time, DOE told NRC that the performance indicators for the panel 
were revised to reflect the change in the work as the project moved 
into the engineering, procurement, and construction phase. DOE also 
reduced the total number of visible indicators from 60 to 30 to focus 
on fewer, more critical aspects of project management. Panels with the 
new indicators were then produced for 6 months, starting with March 
2005 and ending after August 2005. This second interruption of the 
panels resulted from another major revision to the indicators; this 
time, indicators are being made congruent with project work as 
designated by DOE's "new path forward," again to focus on fewer, more 
important activities. In December 2005, a senior DOE official told us 
that the project would begin to measure key activities, but without use 
of the panel. 

Trend Evaluation Reports Have Not Specifically Tracked the Initiatives' 
Management Concerns and Have Had Weaknesses Tracking Significant and 
Recurrent Problems for Management Attention: 

According to DOE, some of the Initiatives' areas of concern and their 
associated effectiveness indicators--for example, trends in quality 
problems related to roles and responsibilities--were being captured, at 
least partially, in the project's quarterly trend evaluation reports 
rather than in the performance indicators. However, the trend reports 
are a management tool designed more to identify emerging and 
unanticipated problems than to monitor progress with already identified 
problems, such as those addressed by the Initiatives. In developing 
these reports, trend analysts seek to identify patterns and trends in 
condition reports (CR), which document problematic conditions through 
the project's Corrective Action Program. The trend reports analyze CRs 
for more significant problems (Levels A and B) and minor problems 
(Level C), but not at Level D (opportunities for improvement). The 
trend analysis typically separates the reported problems into 
categories such as organizational unit, type of problem, and cause. 
These categories are intended to provide insights into the problems. 
For example, analysis might reveal that most occurrences of a 
particular type of problem are associated with a certain organization. 

In practice, DOE missed opportunities to use trend reports to call 
attention to progress in the Initiatives' areas of concern. For 
example, the Initiatives sought to clarify roles and responsibilities 
within and between DOE and BSC to ensure clear accountability for 
project results during the project's transition from scientific studies 
to the design and engineering activities necessary to license a 
repository. Similar organizational transition problems were identified 
in the November 2004 trend report. While that report attributed 
increases in the number of causal factors associated with change 
management, supervisory methods, and work organization to recent BSC 
reorganizations and changes in the project from science-based to design 
and engineering activities, it did not specifically mention issues of 
roles and responsibilities or that roles and responsibilities was an 
Initiatives' area of concern. However, an analysis of the cause of the 
problems noted in various significant condition reports, which is 
performed for certain condition reports and outside of the process of 
developing trend reports, found evidence of weaknesses in the 
organizational interfaces among BSC organizations, as well as between 
BSC and DOE. According to this cause analysis, these organizational 
interface weaknesses were associated with some manner of change and 
represented weaknesses in the definition of roles and responsibilities. 
Trend reports are generally based on condition reports, and problems 
with roles and responsibilities seem to be identified in cause analyses 
rather than in the condition reports themselves. 

Similarly, DOE missed an opportunity to use trend reports to discuss 
the Initiatives' goal that the project's line or work organizations 
become more accountable for self-identifying significant problems. The 
August 2005 trend report briefly cited an evaluation of a CR 
highlighting the low rate of self-identification of significant 
problems during the previous quarter and reported the evaluation's 
conclusion that it was not a problem warranting management attention. 
However, the trend report did not mention that about 35 percent of 
significant problems were self-identified during the previous quarter, 
while the Initiatives' goal was that 80 percent of significant problems 
would be self-identified. Thus, the trend report missed an opportunity 
to either raise a performance problem or pose the question of whether 
the Initiatives' goal needed to be reassessed. 

Beyond whether they effectively tracked the Initiatives' areas of 
concern, trend reports face important obstacles, in general, to 
adequately identify recurrent and significant problems: 

* Recurring or similar conditions can be difficult to clearly identify 
for management's attention and resolution. A trend report noted that 
there will be few cases where recurrent conditions are obvious because 
each condition slightly differs. 

* Trend analysis tends to focus on the number of CRs issued, but the 
number of CRs does not necessarily reflect the significance of a 
problem. For example, the number of CRs involving requirements 
management decreased by over half from the first quarter to the second 
quarter of fiscal year 2005. However, this decrease was not a clear 
sign of progress. Not only did the number rise again in the third 
quarter, but the May 2005 trend report also noted that the number of 
all condition reports had dropped during the second quarter. According 
to the report, the volume of CRs in the first quarter had been high 
because of reviews of various areas, including requirements management. 
Another example is the records management problem. The November 2005 
trend report stated that a records management problem identified in 
various CRs, despite accounting for about 50 percent of all business 
administration problems, reflected an underlying error rate of less 
than 1 percent and thus was not a significant problem. 

* The lack of an increasing trend in the number of reported problems 
does not necessarily mean the lack of a significant problem for 
management attention. Knowing the appropriate level of performance, 
regardless of the trend, is difficult without having clearly 
appropriate benchmarks from organizations engaged in activities similar 
to the Yucca Mountain project. Such benchmarks would clarify, for 
example, whether a project's percentages of human performance errors 
compare favorably, regardless of whether the numbers are increasing. 
Similarly, the trend in the number and types of CRs during any period 
is not necessarily a sign of improvement or worsening conditions. 
Trends can be attributed to various factors, including increases in the 
number of audits or self-assessments, which can lead to more CRs being 
issued. 

* At the time of analysis, some trend data may not be sufficiently 
reliable or complete to ensure sound findings for management's 
attention. For example, although some actions were taken in December 
2004 to ensure that cause and other codes were properly assigned, a BSC 
audit in June 2005 again raised questions about the consistency of the 
coding. With respect to completeness, the fourth quarter report for 
2005 noted that 28 percent of the Level B CRs did not have a cause code 
at the time of the trend analysis, and one finding was presented even 
though two-thirds of the data was missing. 

Due, in part, to these obstacles and changes to how the analysis is 
done, trend reports have not consistently determined the significance 
of problems or performed well in tracking progress in resolving 
problems. For example, trend reports have questionably identified 
significant human performance problems and ineffectively tracked 
progress in resolving the problem because of no clearly appropriate or 
precise benchmark for performance, inconsistent focus on the problem, 
and unreliable data on cause codes. 

The February 2004 trend report identified a human performance problem 
based on Yucca Mountain project data showing the project's proportion 
of skill-based errors to all human performance errors was two times 
higher than benchmark data from the Institute of Nuclear Power 
Operations (INPO).[Footnote 11] The report used this comparison to 
suggest that the project needed to adopt successful commercial nuclear 
practices for addressing skill-based errors. However, the report 
cautioned that other comparisons with these INPO data may not be 
appropriate because of differences in the nature, complexity, and scope 
of work performed, but did not explain why the report's comparison of 
INPO data for skill-based errors to the Yucca Mountain project should 
be an exception to this caution. The May 2004 trend report repeated 
this comparison to INPO, finding skill-based errors three times higher 
than the benchmark data. However, this INPO benchmark has not been used 
in subsequent reports. 

The November 2004 trend report redefined the problem as the 
predominance of human performance errors in general, rather than the 
skill-based component of these errors--but later reports reinterpreted 
this predominance as not a problem. The problem with skill-based errors 
was unclear in the November 2004 report because these errors were 
showing a decreasing trend, a finding that was attributed as likely the 
result of unreliable assignment of cause codes. Instead, the report 
cited an adverse trend based on the fact that the human performance 
cause category accounted for over half of the total number of causes 
for condition reports prepared during the quarter. Under the project's 
trend analysis guidelines, this large predominance of human performance 
causes--in contrast to management, communication or procedure, and 
other cause categories--was designated an adverse trend. Nevertheless, 
by February 2005, trend reports began interpreting this predominance as 
generally appropriate, given the type of work done by the project. That 
is, the project's work involves mainly human efforts and little 
equipment, while work at nuclear power plants involves more 
opportunities for errors caused by equipment. In our view, this 
interpretation that a predominance of human performance errors would be 
expected implies an imprecise benchmark for appropriate performance. 

Although trend reports continued to draw conclusions about human 
performance problems, the February 2005 report indicated that any 
conclusions were hard to justify because of data reliability problems 
with cause coding. For example, the majority of problems attributed to 
human performance causes are minor, or Level C, problems that receive 
less rigorous cause analysis, such as not completing a form. This less 
rigorous analysis tends to reveal only individual human errors--that 
is, human performance problems--whereas more rigorous analysis tends to 
reveal less immediately obvious problems with management and 
procedures. 

Trend reports have also inconsistently tracked progress in resolving 
the problem associated with the "flow-down" of requirements into the 
project's procedures--that is, with ensuring that program, regulatory, 
and statutory requirements are identified, allocated, and assigned to 
the project organizations that are responsible for applicable 
activities. Such requirements management problems can result in 
inadequate control over design inputs and, possibly, inputs to 
scientific models. Progress with this problem was less clear because of 
inconsistent methods of categorizing requirements management problems 
over time. Initially, based on reviews of annual trends in condition 
reports, the September 2004 and November 2004 trend reports observed a 
systemic and continuing problem in the flow-down of requirements from 
BSC's Project Requirements Document and identified this as an adverse 
trend. In subsequent reports, the requirements flow-down problem was 
variously treated as an aspect of requirements management or records 
management, or as a latent management weakness or weak change 
management. When treated as an aspect of these broader problems, the 
significance of the original flow-down problem and any progress in 
resolving it became diluted and less clear. The primary focus 
eventually became requirements management, which the February 2005 
trend report designated as a potential trend, whereas the flow-down 
problem had earlier been designated an adverse trend. Consequently, as 
a result of this change, the flow-down of requirements got less direct 
attention and analysis--for example, receiving only a footnote in the 
August 2005 trend report stating that the April 2004 condition report 
issued to address the adverse trend was still overseeing implementation 
of corrective actions. 

In addition, because trend reports examine only condition reports 
issued to BSC, they do not always assess the projectwide significance 
of problems such as requirements management.[Footnote 12] When 
analyzing one category of issues associated with requirements 
management, the November 2005 report stated that BSC and DOE shared the 
process problems, which cannot be adequately addressed by just one of 
the organizations. However, for a second category of these issues, the 
report did not analyze most of the condition reports because 6 of the 
10 relevant reports were assigned to DOE. For a third category of 
issues, no analysis or recommendation was provided because all of the 
reports were assigned to DOE and therefore did not fall within the 
scope of the trend report. 

DOE Has Not Adequately Tracked Problems Being Addressed by Ongoing 
Management Actions: 

The tracking of problems for which corrective actions are already being 
taken appeared at times to be overly influenced by judgments, rather 
than the problems' significance, about whether additional management 
action is warranted. As a result, problems might be rated as less 
significant, or not tracked further. 

The situation of assigning a lower rating to a problem's significance 
was apparently caused by the fact that ratings were simultaneously an 
assessment of a problem's significance and of the need for management 
action. In its current formulation, DOE's rating categories cannot 
accurately represent both the assessment of a problem's significance 
and a judgment that additional actions are not needed because the 
designated rating category will distort one or the other. For instance, 
the November 2005 trend report analyzed the four categories of 
requirements management issues and designated one category that 
included problems with requirements flow-down as a "monitoring trend"-
-defined as a small perturbation in numbers that does not warrant 
action but needs to be monitored closely. Describing this trend as a 
small perturbation, or a disturbance in numbers, did not accurately 
describe the report's simultaneous recognition that significant process 
problems spanned both BSC and DOE and the fact that the numbers and 
types of problems were consistently identified over the previous three 
quarters. A more understandable explanation for the low rating is that 
designating the problem at any higher level of significance would have 
triggered guidelines involving the issuance of a condition report, 
which, according to the judgment expressed in the report, was not 
needed. Specifically, the report indicated that existing condition 
reports have already identified and were evaluating and resolving the 
problem, thereby eliminating the need to issue a new condition report. 

By rating the problem at the lowest level of significance and not 
calling for additional actions, the trend report did not sufficiently 
draw management's attention to the problem. The trend report's 
assessment did not convey that other serious problems might have been 
raised by the additional condition reports. At about the same time that 
the trend report judged that no new condition reports were necessary, 
an Employee Concerns Program's investigation of requirements management 
resulted in 14 new condition reports--3 at the highest level of 
significance and 8 at the second-highest level of 
significance.[Footnote 13] For example, the Employee Concerns Program's 
investigation resulted in condition reports calling for an analysis of 
the collective significance of the numerous existing condition reports 
and an assessment of whether the quality assurance requirement for 
complete and prompt remedial action had been met.[Footnote 14] As a 
result of the investigation and a concurrent DOE root cause analysis, 
during the December 2005 Quarterly Management Meeting with NRC, DOE 
stated that strong actions were required to address the problems with 
its requirements management system and any resulting uncertainty about 
the adequacy of its design products.[Footnote 15] 

Trend reports identified significant problems in the February 2005 
report but did not continue to track the problems after a separate 
analysis identified ongoing improvement actions. According to the trend 
report, Level B condition reports collectively indicated organizational 
weaknesses associated with change management involving cross- 
departmental interfaces. The trend report recommended that management 
focus on these problems, and cited a condition report that would 
further investigate them. The cause analysis for that condition report 
and a related condition report found that the problems were well-known, 
in part through a BSC review, and related to a variety of ongoing BSC 
improvement actions. Since this was a broad category of problems with 
many initiatives under way, the cause analysis recommended no new 
actions other than for management to remain aware of the problems. 
However, the trend reports that followed provided no further analyses 
to focus management's awareness on these problems or to assess progress 
in resolving them. 

DOE's 'New Path Forward' for Preparing to Submit Its License 
Application Faces Substantial Quality Assurance and Other Challenges: 

In October 2005, DOE announced an aggressive series of proposed changes 
to the design, organization, and management of the Yucca Mountain 
project, but this effort--known as the "new path forward"--will face 
substantial challenges. Some key challenges facing DOE are (1) 
determining the extent of problems and restoring confidence in the 
documents supporting the license application after the discovery of e- 
mails raising the potential of falsified records, (2) settling design 
issues and associated problems with requirements management, and (3) 
replacing key personnel and managing the transition of new managers and 
other organizational challenges. The current Acting Director of the 
Office of Civilian Radioactive Waste Management (OCRWM) stated that DOE 
will not announce a schedule for submitting a license application until 
DOE addresses these important quality assurance and other challenges. 
Since DOE is still formulating its plans, it is too early to determine 
whether the new path will resolve these challenges. 

Determining the Extent of Problems with Relevant Documents Will Delay 
DOE's Submission of the License Application: 

In March 2005, after announcing the discovery of USGS e-mails 
suggesting the possible violation of quality assurance requirements, 
including the falsification of records, DOE has taken steps to address 
lingering concerns about the adequacy of the scientific work related to 
the flow of water into the repository and whether similar quality 
assurance problems are evident in other e-mails relevant to the 
licensing application. Specifically, DOE is (1) conducting an extensive 
review of approximately 14 million e-mails to determine whether these e-
mails raise additional quality assurance concerns and whether they 
might be relevant to the licensing process, and (2) reworking the 
technical documents created by USGS personnel to ensure that the 
science underlying the conclusions on water infiltration are correct 
and supportable in the license application. The Acting Director of 
OCRWM has stated that DOE will not submit a license application until 
these efforts are complete. Consequently, given the early planning 
stage of these efforts, it is unknown how long this will delay the 
submission of a license application. 

As part of the licensing process, DOE is required to publicly disclose 
all documents relevant to the licensing application, including e-mails, 
by posting them on DOE's public Web site, which is accessible through 
the NRC-sponsored, Internet-based Licensing Support Network (LSN). To 
satisfy schedule requirements, DOE must certify that relevant documents 
have been posted to the network and made available for public review 6 
months before the submission of the license application. In preparation 
for submitting the license application by December 2004, in June of 
that year, DOE submitted almost 700,000 e-mails to the LSN that had 
been reviewed by their original authors and determined to be relevant 
to the licensing process. They were part of a group of approximately 6 
million archived e-mails authored by individuals still associated with 
the project. However, in August 2004, NRC's Atomic Safety and Licensing 
Board ruled that DOE had not met its regulatory obligation to make all 
relevant documentary material available. Specifically, DOE had not 
reviewed a group of approximately 4 million archived e-mails authored 
by individuals no longer affiliated with the project to determine 
whether the e-mails were relevant to the licensing process. As part of 
its effort to address the board's ruling, BSC began a review of e-mails 
authored by employees who were not currently working at the project. 
During this review, the contractor discovered and brought forward e- 
mails between USGS scientists working on water infiltration models that 
raised questions of the potential falsification of technical 
information in order to sidestep quality assurance requirements. 

Following the discovery of the e-mails, DOE conducted a search to 
determine if there were similar e-mails in the approximately 1 million 
e-mails previously determined relevant for licensing. However, the DOE 
Inspector General reported in November 2005[Footnote 16] that there was 
no evidence that the project requirements for identifying and 
addressing conditions adverse to quality, such as those contained in 
the USGS e-mails, were considered during the initial review of e-mails. 
Further, among the approximately 10 million e-mails that had already 
been reviewed for the licensing process, they found additional e-mails 
that identified possible conditions adverse to quality that had not 
been identified by project personnel as requiring further review. The 
DOE Inspector General recommended, among other things, that DOE (1) 
expand the review of archived e-mails to include both those deemed 
relevant and those deemed not relevant to the licensing process, and 
ensure that conditions adverse to quality are appropriately identified, 
investigated, reported, and resolved; and (2) ensure that current and 
future e-mails are reviewed for possible conditions adverse to quality 
and that such conditions are appropriately addressed under the 
Corrective Action Program (CAP) system. DOE accepted the Inspector 
General's recommendations. Specifically, DOE agreed to develop a 
corrective action plan to expand the review of archived e-mails to 
ensure that conditions adverse to quality are appropriately identified 
and processed under the CAP system. In addition to this review, the DOE 
Inspector General opened a criminal investigation into the USGS e-mails 
in March 2005. As of December 2005, the investigation was still in 
progress. 

According to NRC on-site representatives, completing these e-mail 
reviews will be challenging because DOE now has to screen millions of e-
mails to ensure that records were not falsified. Further, many of these 
e-mails were written by employees who no longer work at the project or 
may be deceased, making it difficult to learn their true meaning and 
context. Moreover, if additional e-mails are found that raise quality 
assurance concerns, DOE may have to initiate further review, 
inspections, or rework to address the newfound problems. NRC officials 
stated that it takes the issue of potentially falsified documents by 
USGS employees very seriously, wants a full understanding of the 
situation regarding the USGS e-mails, and will conduct follow-up in 
this area. Because NRC wants DOE to submit a high-quality license 
application, it has encouraged DOE to take the time and actions 
necessary to fully and adequately resolve these and other quality 
assurance issues. 

Immediately following the discovery of the USGS e-mails, DOE undertook 
a scientific investigation into the technical documents created by USGS 
personnel. In October 2005, DOE began developing an action plan for 
reviewing, validating, augmenting, and replacing USGS work products 
that had come under scrutiny. Although the plan is not yet complete, 
the Acting Director told us that the license application would not be 
submitted until the USGS work is replaced and there is confidence that 
all requirements have been met. In an effort to ensure that the 
scientific work underlying water infiltration modeling is accurate, DOE 
is working to corroborate the original work by engaging multiple 
agencies and organizations to rework the models. For example, DOE has 
(1) had its lead project contractor work with the Idaho National 
Laboratories to extensively review the software and data used in the 
original science work, (2) engaged Sandia National Laboratories to 
rework the model and calculations using different software than was 
used originally, and (3) also asked USGS to rework the models. 
Consequently, when this additional rework is completed, DOE will have 
four sets of analysis (including the original scientific work) with 
which they can evaluate, compare, and corroborate results. DOE will 
then pick one set of scientific analysis for inclusion in the license 
application, and work to explain and defend its choice. 

Ongoing Design and Requirements Management Issues Could Delay DOE's 
Submission of the License Application: 

In October 2005, DOE announced significant changes to the design of the 
Yucca Mountain repository to simplify the project and improve its 
safety and operation. However, these changes will also require 
additional design and engineering work that will add uncertainty about 
the timing of the submission of a license application. DOE had been 
considering a design where radioactive waste would be shipped to the 
Yucca Mountain site, removed from its shipping container, placed and 
sealed in a special disposal container, and finally moved into the 
underground repository. As a result, DOE contemplated handling the 
waste up to four separate times. In late 2003, DOE engineers began 
identifying potential safety problems with this approach. First, 
possible fissures or holes in the cladding surrounding the spent 
nuclear fuel accidentally caused during the handling of the waste could 
cause air to mix with the fuel and oxidize. Consequently, this 
radioactive oxidized material could then leak and be dispersed into the 
air. Second, DOE engineers determined that the original facility design 
would not be able to adequately control the levels of radioactivity in 
the buildings where the waste would be repackaged before being moved in 
the repository. To address these problems, DOE researched a series of 
options, including only accepting radioactive waste that had already 
decayed to the point where oxidization would not be problematic, and 
testing the waste shipments for oxidization and treating them at 
another site before they arrived at the repository. In addition, DOE 
also considered changing the design by filling the processing buildings 
with inert gas to prevent oxidization and revising the electrical and 
ventilation systems. According to a DOE official, these options were 
impractical or added complexity to the design. 

However, in October 2005, DOE proposed a new design that relies on 
uniform canisters that would be filled and sealed before being shipped, 
eliminating the need for direct handling of the waste prior to being 
placed in the repository. As a result, DOE will not have to construct 
several extremely large buildings costing millions of dollars for 
handling radioactive waste. DOE believes this change will improve the 
safety, operation, and long-term performance of the repository. 
However, this change will also pose a challenge to the project because 
of the widespread implications and the unknown time and effort required 
to implement it. For example, to implement the new design, DOE will 
need to, among other things, 

* get approval from the Energy Systems Acquisition Advisory 
Board[Footnote 17] for a new project plan, which, among other things, 
includes details on the conceptual design, cost estimates, risk 
management efforts, and acquisition strategies; 

* plan, design, and produce standardized canisters for the 
transportation of waste; 

* coordinate this new approach with commercial nuclear power plants, 
NRC, and government organizations that plan on shipping waste to the 
project; and: 

* revise procurement and contracting plans to support the new design. 

Finally, DOE will need to perform the detailed design and engineering 
work required to implement the new design, and create new technical 
documents to support the license application. However, before it can 
present its new plans and perform this design and engineering work, DOE 
officials have stated that it will need to resolve long-standing 
quality assurance problems involving requirements management. 
Requirements management is the process that ensures the broad plans and 
regulatory requirements affecting the project are tracked and 
incorporated into specific engineering details. According to DOE's root 
cause analyses, low-level documents were appropriately updated and 
revised to reflect high-level design changes through fiscal year 1995. 
However, from 1995 through 2002, many of these design documents were 
not adequately maintained and updated to reflect current designs and 
requirements. Further, a document that is a major component of the 
project's requirements management process was revised in July 2002, but 
has never been finalized or approved. Instead, the project envisioned a 
transition to a new requirements management system after the planned 
submission of the license application in December 2004. However, for 
various reasons, the license application was not submitted at that 
time, and the transition to a new requirements management system was 
never implemented. As a result, the document refers to the out-of-date 
NRC regulations contained in 10 CFR part 60, and not the regulations in 
10 CFR part 63 that were finalized in October 2002. 

The scope and cause of requirements management problems have been 
identified in multiple DOE and NRC reviews. 

* Multiple condition reports issued in 2004 and 2005 have identified 
problems with requirements management. Due to these condition reports 
and NRC concerns that repetitive deficiencies and the failure to 
implement timely corrective actions could have direct implications on 
the quality of the planned license application, NRC performed a review 
of Corrective Action Program documents related to the requirements 
management program in the late summer of 2005. NRC determined that 
these reports identified approximately 35 deficiencies related to 
requirements management. Because the requirements management documents 
are not current and the new requirements management system has not been 
implemented, NRC concluded that there does not appear to be a 
requirements management mechanism in place. Further, based on the 
number of reports and other issues identified by DOE audits, NRC 
concluded that the project's Corrective Action Program was not 
effective in, among other things, eliminating the repeated 
identification of deficiencies relating to requirements management or 
initiating the actions to identify and appropriately address the root 
cause of these problems. 

* In September 2005, DOE began reviewing the root causes associated 
with CR-6278, a condition report identifying problems with requirements 
management. As part of the review, DOE personnel analyzed 135 condition 
reports and other events and allegations. Among other things, this 
review found that DOE expectations for requirements management were 
diluted and eventually neglected, that DOE reduced funding for 
requirements management due to reductions in its annual budget, and 
that these and other events caused the requirements management process 
to become "completely dysfunctional" from July 2002 to the time of the 
review in the fall of 2005. The analysis identified the root causes of 
these conditions as DOE's failure to fund, maintain, and rigidly apply 
a requirements management system. 

* In November 2005, a team of DOE personnel concluded an investigation 
into an employee's concerns regarding requirements management. The team 
substantiated all of the concerns they investigated and found instances 
of failures and breakdowns in the requirements management process. For 
example, among other things, the team found that no procedure was 
developed to describe how requirements management was to occur; some 
existing requirements management procedures were not implemented; and 
project management was aware of these conditions but corrective actions 
were deferred because the planned requirements management system was 
expected to address the problem. As a contributing factor, the team 
also observed that the project's lead contractor had not implemented a 
"traditional systems engineering approach" as it did not have, among 
other things, typical engineering management plans or a separate 
systems engineering organization responsible for requirements 
management. As a result of the investigation, the team initiated 14 
condition reports, 13 of which identified quality-related problems. 

To address these problems, on December 19, 2005, DOE issued a stop-work 
order on design and engineering for the surface facility and certain 
other technical work. DOE stated that the root cause analysis for CR 
6278 and the investigation into employee concerns revealed that the 
project has not maintained or properly implemented its requirements 
management system, resulting in inadequacies in the design control 
process. This stop-work order will be in effect until, among other 
things, the project's lead contractor improves the requirements 
management system; validates that processes exist and are being 
followed; and requirements are appropriately traced to implementing 
mechanisms and products. Further, DOE will establish a team to take 
other actions necessary to prevent inadequacies in requirements 
management and other management systems from recurring. 

An example of the potential risks of a breakdown with requirements 
management was noted during a BSC audit on the design process in March 
2005. NRC on-site representatives observing this audit reported that 
the audit team noted problems with inconsistencies between the design 
documents of the planned fuel-handing facility that would be receiving, 
preparing, and packaging the waste before it is placed in the 
repository. The original set of requirements specified that no water 
from a fire protection system was to be used in the fuel-handling areas 
of the facility because under certain scenarios, water used for fire 
suppression could facilitate an accidental nuclear reaction, a 
condition known as criticality. Later, as the project began to review 
the design of the fuel-handling facility, the design was changed to 
allow the use of water sprinklers in the fuel-handling areas of the 
facility to suppress possible fires. NRC noted that personnel working 
on the design knew of the inconsistencies between older and newer 
design documents, but no formal tracking mechanism had been provided to 
ensure that those issues were rectified. According to an NRC on-site 
representative in December 2005, this was an example of a concern with 
requirements management, and that repetitive and uncorrected issues 
associated with the requirements management process could have direct 
implications on the quality of the license application. 

While the project may be able to resolve these inconsistencies through 
an informal process, the lack of a formal design control and 
requirements management process increases the risk that not all such 
problems will be addressed. These requirements management problems are 
potentially significant because if the high-level engineering needs of 
the project are not accurately or completely reflected in the detailed 
design, then the quality of the license application may be compromised 
and cause delays in the license application review process. For 
example, according to a 1989 speech prepared by NRC's Office of General 
Counsel stressing the importance of quality assurance, a West Coast 
nuclear power plant experienced similar quality assurance problems with 
requirements management. After a license was issued by NRC, power plant 
personnel discovered that the wrong diagrams were used to develop 
design requirements. As a result of this and other quality assurance 
weaknesses identified by NRC, the license was suspended and the power 
plant was required to initiate an independent program to verify the 
correctness of the design. Further, NRC reopened hearings on the issue 
of the adequacy of the power plant's quality assurance program related 
to the plant's design. 

DOE Faces Challenges in Managing the Transition, Complexity, and 
Continuity of Its 'New Path Forward': 

In October 2005, DOE announced a "new path forward" that would create a 
new project schedule and financial plan to address the completion of 
scientific and engineering work in support of a license application. 
However, DOE faces challenges to successfully implementing the new 
path, in terms of managing the transition, program and organizational 
complexities, and the continuity of management. According to DOE 
managers involved with planning the new path forward, the 
organizational transition could take several months to complete. It is 
too early to determine whether DOE's new effort will resolve quality 
assurance issues and move the project forward to the submission of a 
license application. 

Accountability for quality and results, which was identified as a 
significant transition issue in the Initiatives, will likely pose a 
challenge for managing the transition to the new path forward. The 
Initiatives sought to clarify roles and responsibilities within and 
between DOE and contractor organizations to ensure clear accountability 
for results and quality during the transition from OCRWM's 
organization, processes, procedures, and skills supporting scientific 
studies to those supporting the activities necessary to license a 
repository. As the project realigns organizations, processes, 
procedures, and skills to support the new path forward, it will also be 
faced with the challenge of ensuring that accountability is not 
undermined during the transition. For instance, according one DOE 
manager, transitioning project work to a lead laboratory under a direct 
contract with DOE could pose a significant challenge for quality 
assurance because the laboratories are currently working under BSC 
quality assurance procedures and will now have to develop their own 
procedures. 

Implicitly recognizing the importance of accountability issues, 
elements of the new path forward seek to address issues that can 
negatively affect quality assurance and project management in general. 
For instance, the new path includes plans for developing and 
transmitting requirements to USGS for the certification of scientific 
work. In addition, a senior project official told us that the lead 
laboratory would provide a single point of accountability that will 
enhance the quality of the science work. The Acting Director indicated 
that OCRWM's management structure may have to be reorganized to have a 
single manager clearly accountable for each of the new path's major 
tasks in science, engineering, and licensing. Moreover, the project is 
developing new performance indicators to allow the project to assess 
important activities under the new path forward. Outside of the new 
path, as the result of a September 2005 DOE Inspector General 
report[Footnote 18] on accountability problems with managing contract 
incentives, OCRWM agreed to develop a comprehensive corrective action 
plan to provide clearer and more objective performance standards in the 
BSC contract. 

Program complexity and other project characteristics are also likely to 
pose challenges to managing quality assurance. Based on its experience 
with licensing and regulating nuclear power plants, NRC observed in the 
mid-1980s that the Yucca Mountain project's characteristics, such as a 
large and complicated program, increased the likelihood of major 
quality-related problems. Although the new path is intended to simplify 
design, licensing, and construction, the project remains a complicated 
program that seeks to both restore confidence in its scientific studies 
and pursue new design and engineering activities. As a result, the 
project has to manage quality assurance issues simultaneously in both 
areas. Moreover, the project involves a complicated organizational 
structure. The project will continue contracting work with BSC, USGS, 
and the Sandia National Laboratory, which involves working with 
organizations in various locations. In our 1988 report, we noted that 
the geographic distance between the various organizations may hamper 
OCRWM's quality assurance communication and oversight objectives. 

The project also faces management challenges related to ensuring 
management continuity at the project, since DOE has experienced 
turnover in 9 of 17 key management positions since 2001. To ensure the 
right managers move the project forward to licensing, the project has a 
recruitment effort for replacing key departing managers. In the past 
year, the project has lost key managers through the departures of the 
director of Project Management and Engineering, the director of the 
License Application and Strategy, the director of Quality Assurance, 
and the contractor's general manager. According to NRC on-site 
representatives in August and October 2005, management turnover is a 
concern for NRC because it would like to see continuity of qualified 
managers rather than a series of acting managers. Recruiting 
replacement managers can impact project continuity, and newly acting 
managers may not take full rein of project tasks. However, the Acting 
Director told us that the recruitment process is an opportunity to 
improve project managers and staff, but recruiting the right people is 
challenging for various reasons--for example, government salaries are 
less than those in industry, and employment clauses restrict subsequent 
employment in related industries. 

Finally, since new directors sometimes give new direction to the 
project, a critical issue for sustaining the current new path forward 
is continuity with OCRWM's director. This position was occupied by 
three individuals between late 1999 and early 2005. The last OCRWM 
director assumed the position in April 2002, started the Management 
Improvement Initiatives in 2002, and left the position in February 
2005. The current Acting Director began functioning in his position in 
the summer of 2005, and initiated the new path forward in October 2005. 
DOE is currently awaiting congressional confirmation of a nominee to 
take the director position. However, the Acting Director told us he 
expects that the new path forward will be sustained because it has been 
endorsed by the Secretary of Energy. 

Conclusions: 

DOE's Yucca Mountain project has been wrestling with quality assurance 
problems for a long time. Now, after more than 20 years of project 
work, DOE is again faced with substantial quality assurance and other 
challenges to submit a fully defensible license application to NRC. 
Unless these challenges are effectively addressed, further delays on 
the project are likely. Furthermore, even as DOE faces new quality 
assurance challenges, it cannot be certain it has resolved past 
problems, largely because the department has not been well served by 
management tools--specifically, its performance indicators and trend 
evaluation reports--that have not effectively identified and tracked 
progress on significant and recurring problems. First, the management 
tools have provided limited coverage of the areas of concern identified 
in the Management Improvement Initiatives and thus have not enabled DOE 
managers to effectively monitor progress in these important areas. 
Second, the tools have often not reflected the full extent or 
significance of problems because their scope has been limited and not 
based on projectwide analysis. Third, the trend evaluation reports 
have, at times, not accurately characterized problems because reliable 
and complete data and appropriate performance benchmarks were not 
available at the time of analysis. Fourth, frequent changes in 
performance indicators and the way analysis is done have made it 
difficult to accurately identify trends over time. Fifth, the tools' 
rating categories have sometimes been misleading as to the significance 
of problems because the ratings tend to be skewed by the fact that 
corrective actions were already being taken, without considering their 
effectiveness or considering the significance of the problem on its own 
terms. These shortcomings with the tools limit project managers' 
ability to direct and oversee such a large and complex undertaking as 
constructing an underground repository for nuclear wastes. Further 
complicating DOE's ability to manage the project are the vacancies in 
key managerial positions for the quality assurance program and 
elsewhere on the project. The tools become even more important for new 
managers who need to quickly understand project management issues. 

Recommendations for Executive Action: 

To improve the effectiveness of DOE's efforts to monitor performance in 
key areas at the Yucca Mountain project, including quality assurance, 
we recommend that the Secretary of Energy direct the Director, Office 
of Civilian Radioactive Waste Management, to take the following five 
actions to strengthen the project's management tools: 

* Reassess the coverage that the management tools provide for the areas 
of concern identified in the Management Improvement Initiatives and 
ensure that performance in these important areas is effectively 
monitored, especially in light of the more recent condition reports and 
associated cause analyses, trend reports, and other reviews indicating 
continuing problems. 

* Base future management tools, such as the trend evaluation reports, 
on projectwide analysis of problems, unless there are compelling 
reasons for a lesser scope. 

* Establish quality guidelines for trend evaluation reports to ensure 
sound analysis when reporting problems for management's attention. Such 
guidelines should address, among other things, having reliable and 
complete data and appropriate benchmarks. 

* To the extent practicable, make analyses and indicators of 
performance consistent over time so that trends or progress can be 
accurately identified and, where changes to analyses or indicators are 
made for compelling reasons, provide a clear history of the changes and 
their impact on measuring progress. 

* Focus the management tools' rating categories on the significance of 
the monitored condition, not on a judgment of the need for management 
action. 

Agency Comments: 

We provided DOE and NRC with draft copies of this report for their 
review and comment. In oral comments, DOE agreed with our 
recommendations and provided technical and editorial comments that we 
have incorporated in the report, as appropriate. We also incorporated, 
as appropriate, NRC's oral editorial comments, which primarily served 
to clarify its role. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to interested 
congressional committees and Members of Congress, the Secretary of 
Energy, and the Chairman of the Nuclear Regulatory Commission. We will 
also make copies available to others upon request. In addition, the 
report will be available at no charge on the GAO Web site at 
[Hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-3841 or [Hyperlink, wellsj@gao.gov]. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix III. 

Sincerely yours, 

Signed by: 

Jim Wells: 
Director, Natural Resources and Environment: 

[End of section] 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

The objectives of this review were to determine (1) the history of the 
Yucca Mountain project's quality assurance problems since the project's 
start in the 1980s, (2) the Department of Energy's (DOE) tracking of 
quality problems and progress implementing quality assurance 
requirements since our April 2004 report, and (3) challenges that DOE 
faces as it continues to address quality assurance issues within the 
project. In addition, we were asked to provide information about 
implementation of the project's Employee Concerns Program and the types 
of concerns raised in recent years through the program. 

To determine the history of the project's quality assurance problems, 
we reviewed our prior reports and those of DOE's Office of the 
Inspector General concerning the Yucca Mountain project. We also 
reviewed internal DOE evaluations and audit reports written about the 
quality assurance program and Nuclear Regulatory Commission (NRC) 
reports and NRC-prepared summaries of NRC and DOE quarterly management 
meetings, technical exchange meetings, and quality assurance meetings 
dating to early 2004. In addition, we reviewed letters and 
communications between DOE and NRC regarding quality assurance from the 
NRC Web archives from the late 1980s. Furthermore, we reviewed plans 
for the Regulatory Integration Team (RIT) and subsequent correspondence 
between Bechtel/SAIC Company, LLC (BSC), DOE's management contractor 
for the Yucca Mountain project, and DOE. Moreover, we discussed quality 
assurance issues with officials of DOE's Office of Civilian Radioactive 
Waste Management (OCRWM), including the Acting Director and Deputy 
Director, at DOE headquarters in Washington, D.C., and at its field 
office in Las Vegas. In addition, we interviewed representatives of 
Navarro Quality Services, a DOE subcontractor, as well as BSC, and NRC 
officials in the agency's field office in Las Vegas, Nevada, and at its 
headquarters in Rockville, Maryland. 

To determine DOE's tracking of quality problems and progress 
implementing quality assurance requirements since our April 2004 
report, we interviewed OCRWM, BSC, and NRC officials about the status 
of these efforts since the issuance of our prior report. We also 
reviewed DOE's Management Improvement Initiatives (2002), DOE's 
Management Improvement Initiatives Transition Approach (2003), and our 
2004 report to understand the history of the improvement efforts. To 
understand DOE's management tools to monitor problems and progress, we 
reviewed the available performance indicators panels from April 2004 
through August 2005, when it was last produced; the documentation on 
the individual indicators applied to August 2005 data; and the 
quarterly trend reports from the fourth quarter of fiscal year 2003 
through the fourth quarter of fiscal year 2005. We also reviewed 
information from condition reports and examined documentation on DOE's 
Quality Assurance Requirements and Description (issued in August 2004), 
BSC's Trend Evaluation and Reporting, and DOE's Procedure: Condition 
Reporting and Resolution (issued in November 2005). 

To determine challenges that DOE faces as it continues to address 
quality assurance issues within the project, we reviewed information 
from condition reports, NRC on-site representative reports, DOE 
Inspector General reports, and an OCRWM Office of Concerns Program's 
investigative report on past quality assurance problems and DOE's 
efforts to address them. We obtained information on turnover in key 
management positions at DOE and BSC since 2000. In addition, we 
discussed with DOE and NRC officials DOE's difficulties in addressing 
recurring quality assurance problems and the quality assurance 
implications of the Yucca Mountain project moving from the site 
characterization phase to design and licensing. Also, to better 
understand issues and challenges, we attended quarterly meetings held 
between DOE and NRC in Rockville in September 2005 and Las Vegas in 
December 2005. 

To identify recent employee concerns related to quality assurance, such 
as falsification of records and a safety-conscious work environment, as 
well as to identify the actions taken to address those concerns, we 
reviewed all concerns received by the OCRWM and BSC Employee Concerns 
Programs from January through November 2005. For the OCRWM program, we 
reviewed all employee concerns files to identify concerns related to 
quality assurance. For the BSC program, we first read summary 
descriptions of each concerns file, and reviewed the concerns files for 
only those we identified as related to quality assurance. We then 
conducted a content analysis of all concerns files that we reviewed. 
Next, our three team members reached consensus about the correct 
classification of a concern as a quality assurance problem, such as 
potential falsification of records. Finally, through a second review of 
concerns files, we verified our recorded information for those concerns 
that seemed to be important illustrations of problems. In addition, we 
also spot-checked a sample of OCRWM and BSC concerns received in 2005 
to verify the accuracy of their placement in various concerns 
categories. We found that the concerns were generally categorized 
accurately. 

We performed our work from July 2005 through January 2006 in accordance 
with generally accepted government auditing standards. 

[End of section] 

Appendix II: Yucca Mountain Project Employee Concerns Programs: 

NRC expects licensees to establish a safety conscious work environment-
-that is, one in which (1) employees are encouraged to raise concerns 
either to their own management or to NRC without fear of retaliation 
and (2) employees' concerns are resolved in a timely and appropriate 
manner according to their importance. NRC encourages but does not 
require licensees to establish employee concerns programs to help 
achieve such a work environment, and both DOE and BSC have established 
such programs.[Footnote 19] DOE's Employee Concerns Program is 
currently operated under the requirements of DOE Order 442.1A, but the 
department, in anticipation of becoming a licensee, is in the process 
of establishing the program to meet NRC expectations. 

DOE and contractor employees at the Yucca Mountain project may raise 
concerns about quality, safety, or other work environment issues--such 
as harassment, intimidation, retaliation, and discrimination--through 
various means. Employees are encouraged to resolve concerns at the 
lowest possible level in the organization, in the following order: 

* Use normal supervisory channels, such as by raising an issue to a 
manager for resolution. 

* Initiate a condition report through the Corrective Action Program--a 
process in which any employee can formally identify a problem on the 
project, such as with policies, procedures, or the work environment, 
and have the issue investigated and, if necessary, fixed through 
corrective actions. 

* Submit a concern via e-mail, telephone, or in person to one of the 
project's two Employee Concerns Programs--a BSC program for BSC 
employees and other subcontractors and another run by DOE for either 
DOE or BSC employees. 

* Contact NRC directly. 

The DOE and BSC concerns programs are intended to supplement rather 
than replace the resolution of problems through managers or the 
Corrective Action Program. 

DOE and BSC Employee Concerns Programs have each established a 
communication network to allow employees to register concerns. These 
networks include brochures and regular newsletters on the programs and 
numerous links to the program on the project's intranet, where 
employees can obtain concerns forms. Both the DOE and BSC concerns 
programs of the Yucca Mountain project have four main steps: 

1. Employees notify the concerns program staff about issues that they 
feel should be corrected, such as safety or health issues; harassment, 
intimidation, retaliation, or discrimination; concerns raised through 
the Corrective Action Program; and quality assurance problems. 

2. The concerns program staff document and handle the concern in 
accordance with the requirements of DOE Order 442.1A. 

3. The concerns program notifies the employees of the results of the 
investigation and notifies management of any deficiencies. 

4. Project management develops corrective actions for deficiencies, and 
the program validates that the concerns have been effectively addressed 
by the actions. 

Under DOE Order 442.1A, concerns may be addressed through an 
investigation by the concerns program staff, an independent 
investigation, a referral, a transfer, or a dismissal of the concern. 
Employees can request or waive confidentiality. If a concern is 
submitted anonymously, interpreting the main issues and problems is 
left up to the concerns program staff, and action on the concern may be 
limited if the submitted information does not clearly or sufficiently 
define the concern. 

The concerns program may conduct its own investigation of the concern. 
Alternatively, it may refer the concern to another project organization 
for investigation or resolution. After the results of the investigation 
or resolution are reported to the concerns program within a specified 
period, the concerns program accepts the results or requires additional 
actions. In other cases, concerns may be transferred to another 
organization with the appropriate subject matter responsibility or 
expertise, such as the Office of Human Relations, Office of General 
Counsel, or Office of the Inspector General. 

After investigating a concern, the concerns programs determines whether 
the concern is substantiated, partially substantiated, unsubstantiated, 
or indeterminate. If a concern is substantiated or partially 
substantiated, the investigation results are presented to the 
responsible senior managers. A concern is considered indeterminate when 
evidence is insufficient to substantiate a concern or allow for a 
conclusion to be drawn. Some concerns can be resolved through a 
noninvestigative resolution, a method to address concerns promptly when 
minimal effort is required for resolution. Some resolutions involve the 
development of management corrective action plans that are tracked 
until they are closed. In addition, for deficiencies that identify 
systemic problems, the concerns programs may file a condition report 
through the Corrective Action Program. Moreover, DOE and contractor 
employees are required to report certain conditions or alleged 
conditions to DOE's Office of the Inspector General under DOE Order 
221.1, which covers waste, fraud, and abuse. The concerns program 
handles some employee concerns in this way. 

From January through November 2005, DOE's concerns program opened 139 
employee concerns for investigation, and the BSC concerns program 
opened 112 concerns for investigation.[Footnote 20] DOE's concerns 
program places concerns into 14 categories, while the BSC program uses 
20 categories.[Footnote 21] For both DOE and BSC, the category 
receiving by far the most concerns for calendar year 2005 was 
management: "management/mismanagement" for DOE and "management 
practices" for BSC. According to DOE, management concerns generally 
involved conditions related to management behavior, policy practice, 
budget allocation, or use of resources. According to the manager of 
BSC's program, about half of the concerns in the management practices 
category involve hiring and human relations issues and the other half 
involve organizational policies and other issues. The "quality" 
category accounts for a relatively small portion of total concerns--18 
percent of concerns for the DOE program and 4 percent for the BSC 
program. Tables 3 and 4 show the concerns received by the DOE and BSC 
programs for January through November 2005. 

Table 3: Employee Concerns Opened for Investigation under DOE's 
Employee Concerns Program by Category of Concern, January through 
November 2005: 

Concern category: Management/mismanagement; 
Percentage of total concerns: 42. 

Concern category: Workplace violence; 
Percentage of total concerns: 0. 

Concern category: Harassment, intimidation, retaliation, and 
discrimination; 
Percentage of total concerns: 6. 

Concern category: Reprisal; 
Percentage of total concerns: 0. 

Concern category: Chilling effect; 
Percentage of total concerns: 5. 

Concern category: Security; 
Percentage of total concerns: 0. 

Concern category: Health; 
Percentage of total concerns: 0. 

Concern category: Safety; 
Percentage of total concerns: 4. 

Concern category: Environment; 
Percentage of total concerns: 0. 

Concern category: Fraud, waste and abuse; 
Percentage of total concerns: 4. 

Concern category: Human resources; 
Percentage of total concerns: 12. 

Concern category: Equal Employment Opportunity; 
Percentage of total concerns: 2. 

Concern category: Quality; 
Percentage of total concerns: 18. 

Concern category: Other; 
Percentage of total concerns: 8. 

Concern category: Total; 
Percentage of total concerns: 100. 

Source: DOE. 

Note: Percentages may not add to 100 because of rounding. 

[End of table] 

Table 4: Employee Concerns Opened for Investigation under BSC's 
Employee Concerns Program by Category of Concern, January through 
November 2005: 

Concern category: Management practices; 
Percentage of total concerns: 48. 

Concern category: Industrial; 
Percentage of total concerns: 1. 

Concern category: Health; 
Percentage of total concerns: 4. 

Concern category: Fraud; 
Percentage of total concerns: 3. 

Concern category: Fitness for duty; 
Percentage of total concerns: 1. 

Concern category: Ethics; 
Percentage of total concerns: 5. 

Concern category: Cyber; 
Percentage of total concerns: 0. 

Concern category: Access authorization; 
Percentage of total concerns: 0. 

Concern category: Environmental; 
Percentage of total concerns: 1. 

Concern category: Employee relations; 
Percentage of total concerns: 5. 

Concern category: Intimidation; 
Percentage of total concerns: 1. 

Concern category: Harassment; 
Percentage of total concerns: 0. 

Concern category: Discrimination; 
Percentage of total concerns: 4. 

Concern category: Chilling effect; 
Percentage of total concerns: 4. 

Concern category: Abuse; 
Percentage of total concerns: 4. 

Concern category: Training; 
Percentage of total concerns: 1. 

Concern category: Safety-conscious work environment; 
Percentage of total concerns: 3. 

Concern category: Retaliation; 
Percentage of total concerns: 4. 

Concern category: Quality; 
Percentage of total concerns: 4. 

Concern category: Other; 
Percentage of total concerns: 6. 

Concern category: Total; 
Percentage of total concerns: 100. 

Source: BSC. 

Note: Percentages may not add to 100 because of rounding. 

[End of table] 

The Employee Concerns Programs, which are designed to provide an 
alternative to raising issues through the Corrective Action Program and 
issuing condition reports, have been playing an active and sometimes 
key role in identifying and addressing quality assurance problems, as 
can be seen in the following examples: 

* As part of an effort to identify e-mails relevant to the licensing 
process and that therefore should be included in the Licensing Support 
Network, BSC employees in late 2004 discovered e-mails suggesting 
potential falsification of technical records. The e-mails were 
submitted to the Employee Concerns Program in March 2005 and were 
eventually reported to the DOE Inspector General for investigation. The 
quality assurance issues raised by the e-mails have resulted in a 
substantial effort by DOE to restore confidence in the quality of 
technical documents that will support its license application to 
construct the repository. 

* In mid-2005, the DOE concerns program referred to the project's 
senior management an employee's allegation that the project's schedule 
was taking priority over quality in the review of technical documents. 
In this instance, the Office of Concerns Program Manager negotiated 
with senior management to address the time and resource needs for 
ensuring quality assurance, rather than simply communicating to the 
organization that quality should take priority over the schedule. 

* As the result of an employee's concerns referred to DOE by NRC in mid-
2005, the Employee Concerns Program initiated an extensive 
investigation of issues related to requirements management. That 
investigation substantiated the employee's concerns and led to the 
issuance of 14 condition reports for problem resolution. Signifying the 
importance of this issue, DOE discussed problems with requirements 
management with NRC at their quarterly meeting in December 2005. 

The Employee Concerns Programs' role in identifying and addressing 
quality assurance and other issues is dependent upon employees' 
willingness to submit concerns, but the employees' willingness has 
sometimes been in doubt. A late 2004 DOE survey of project employees 
indicated, for example, that less than two-thirds of employees were 
confident that submitted concerns would be thoroughly investigated and 
appropriately resolved. DOE recognizes the need to improve employee 
trust and willingness to use the concerns program, and both the DOE and 
BSC program are engaged in outreach efforts. However, employees' 
willingness to submit concerns may be affected by factors outside the 
programs' control. According to a DOE manager, the project's recent and 
pending workforce reductions may account for a decreasing number of 
concerns submitted to the DOE program in late 2005. Based on OCRWM 
Employee Concerns Program data, the program averaged about 13 concerns 
a month from January through November 2005. However, the number of 
monthly concerns dropped to 5 in October and 3 in November 2005. 

During our review of concerns opened for investigation from January 
2004 through November 2005, we did not identify any concerns alleging 
problems similar to the falsification of technical records suggested by 
the USGS e-mails. Although we found records of an early 2004 concern 
about an instance of inappropriate management of a technical document, 
this instance was resolved and did not appear to be an intentional or 
systematic effort to falsify records. The manager of the BSC program 
told us of a concern raised about another set of e-mails, but this 
concern was not about record falsification. The manager of the DOE 
program told us that she had not seen any reportable allegations of 
falsification of technical records since she took her position in July 
2004. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Jim Wells, (202) 512-3841 or [Hyperlink, wellj@gao.gov]. 

Staff Acknowledgments: 

In addition to the contact named above, Raymond Smith (Assistant 
Director), Casey Brown, John Delicath, James Espinoza, and Terry 
Hanford made key contributions to this report. 

(360611): 

FOOTNOTES 

[1] Once EPA finalizes its new standards, NRC will revise its licensing 
regulations to make them consistent with the standards. 

[2] GAO, Yucca Mountain: Persistent Quality Assurance Problems Could 
Delay Repository Licensing and Operation, GAO-04-460 (Washington, D.C.: 
Apr. 30, 2004). 

[3] GAO-04-460, 5. 

[4] In addition, DOE must update this certification at the time of 
license application submittal, as required by NRC regulations. 

[5] Docketing is the formal acceptance of the license application by 
NRC after it determines that it contains adequate information for a 
formal review. 

[6] NRC is required to issue or deny a construction authorization not 
later than 3 years after receiving a license application, unless it 
extends this period by not more than 1 year and reporting the reasons 
for doing so to the Secretary of Energy and Congress. 

[7] GAO, Nuclear Waste: Repository Work Should Not Proceed Until 
Quality Assurance Is Adequate, GAO/RCED-88-159 (Washington, D.C.: Sept. 
29, 1988). 

[8] U.S. Nuclear Regulatory Commission, U.S. Nuclear Regulatory 
Commission Staff Evaluation of U.S. Department of Energy Analysis Model 
Reports, Process Controls, and Corrective Actions (Washington, D.C., 
Apr. 7, 2004). 

[9] In addition, the RIT edited the AMRs to assure consistency and ease 
of technical and regulatory reviews. 

[10] Similar to our findings, a January 2006 NRC observation audit 
report noted that a DOE audit had found the trend program was 
unsatisfactory. Specifically, the DOE audit found that the trend 
program was not handled as a priority for management attention, and has 
historically identified broad causal issues with no adverse trends. 

[11] Skill-based errors are defined in trend reports as unintentional 
errors resulting from people not paying attention to the task at hand. 

[12] DOE prepares a separate trend report for CRs issued to DOE, but 
that report's findings are not integrated with those of BSC to provide 
a projectwide analysis. 

[13] The difference in number of CRs issued also reflects the fact that 
the scope of the investigations was broader than the trending report's 
scope, which focused only on CRs assigned to BSC. The investigation 
resulted in CRs assigned to DOE as well as BSC. 

[14] Appendix II offers a detailed description of the employee concerns 
program and the variety of employee submitted concerns, which are often 
not related to quality assurance, and our examination of concerns 
submitted since 2004 that did not find concerns similar to those raised 
about the potential falsification of records by USGS employees. 

[15] A root cause analysis in such reports involves methods for 
determining the root cause of a problem, which is the underlying cause 
that must change in order to prevent the problem from reoccurring. A 
root cause analysis is required for the most significant CRs--those 
determined to be at Level A in the Corrective Action Program. 

[16] U.S. Department of Energy, Office of Inspector General, Office of 
Inspections and Special Inquiries, Quality Assurance Weaknesses in the 
Review of Yucca Mountain Electronic Mail for Relevancy to the Licensing 
Process, DOE/IG-0708 (Washington, D.C., November 2005). 

[17] The Energy Systems Acquisition Advisory Board is a special 
organization within DOE that advises the Secretary in approving and 
revising plans for major acquisition projects. 

[18] U.S. Department of Energy, Office of Inspector General, Office of 
Audit Services, DOE/IG-0702, Use of Performance Based Incentives by the 
Office of Civilian Waste Management (Washington, D.C., September 2005). 

[19] The discussion in this report involves the DOE OCRWM office in Las 
Vegas, Nevada and not the separately operated office in OCRWM 
headquarters in Washington, D.C. The DOE employee concerns program 
oversees BSC's program. 

[20] The number of concerns opened for investigation by BSC includes 
referrals from DOE. Both organizations track disposition of these 
concerns and include them in their totals. According to the manager of 
the BSC Employee Concerns Program, about one-fourth of BSC's concerns 
are DOE referrals. 

[21] We reviewed all of the concerns submitted to DOE and BSC from 
January 2004 through November 2005 for their relevance to quality 
assurance issues. Although we did not verify the accuracy of DOE's and 
BSC's placement of all concerns into the above categories, we spot- 
checked a sample of DOE and BSC concerns and found that they were 
generally categorized accurately. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: