This is the accessible text file for GAO report number GAO-02-890 
entitled 'Chemical Weapons: Lessons Learned Program Generally Effective 
but Could Be Improved and Expanded' which was released on September 10, 
2002.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to Congressional Requesters:



United States General Accounting Office:



GAO:



September 2002:



CHEMICAL WEAPONS:



Lessons Learned Program Generally Effective but Could Be Improved and 

Expanded:



Chemical Weapons:



GAO-02-890:



Contents:



Letter:



Results in Brief:



Background:



Lessons Learned Program Has Made Positive Contributions but Needs 

Improvement:



Sharing of Lessons Learned Could Be Expanded:



Conclusions:



Recommendations for Executive Action:



Agency Comments and Our Evaluation:



Appendix I: Information on the Incineration Process and 

Incidents at Three Sites:



Appendix II: Scope and Methodology:



Appendix III: Lessons Learned Process:



Appendix IV: Chemical Demilitarization Program Management 
Developments, 

1997-2001:



Appendix V: Comments from the Department of the Army:



Tables:



Table 1: Status of the Chemical Stockpile Disposal Project:



Table 2: May 2000 Agent Release at Tooele Chemical Agent Disposal 

Facility:



Figures:



Figure 1: Chem-Demil Programmatic Lessons Learned Program Process:



Figure 2: Lessons Learned Stakeholders and Process Steps:



Figure 3: Chemical Demilitarization Program Organization Chart:



Abbreviations:



DFS: deactivation furnace system



DOD: Department of Defense



GAO: General Accounting Office



LIC: liquid incinerator



SDS: spent decontamination solution:



Letter:



September 10, 2002:



The Honorable Jeff Sessions

The Honorable Gordon Smith

The Honorable Ron Wyden

United States Senate:



The Honorable James V. Hansen

The Honorable Duncan Hunter

The Honorable Bob Riley

House of Representatives:



The Army has been tasked to destroy about 31,500 tons of highly toxic 

chemical agents by April 2007, the deadline set by an international 

treaty for the elimination of all chemical weapon stockpiles. Until 

they are destroyed, the chemical agents will continue to pose a threat 

to the thousands of people living and working near the disposal 

facilities where the agents are being stored. To destroy the weapons, 

the Department of Defense (DOD) established the Army’s Chemical 

Demilitarization (or Chem-Demil) Program. The Army has destroyed over 

one-quarter (8,044 tons) of the U.S stockpile as of March 2002.



Originally, the Chem-Demil Program consisted only of the Chemical 

Stockpile Disposal Project, also known as the baseline incineration 

project, which was initiated in 1988 to incinerate chemical weapons at 

nine storage sites. Then, in response to public concern about 

incineration, Congress established the Alternative Technologies and 

Approaches Project in 1994 to investigate alternatives to the baseline 

incineration process. In 1997, Congress established the Assembled 

Chemical Weapons Assessment Program to identify and test additional 

technologies as alternatives to incineration. Today, five of the nine 

storage sites use incineration; three others will use or plan to use 

alternative technologies. The technology choice for the final site has 

yet to be determined.



The Chemical Stockpile Disposal Project operates a Programmatic Lessons 

Learned Program whose aim is to enhance safety, reduce or avoid 

unnecessary costs, and maintain the incineration schedule. A lesson 

learned is a set of rules or principles that summarizes past 

experiences to help people better perform future tasks. The project’s 

goal is to capture and share lessons learned from experience so that 

stakeholders--engineers, contractors, and program managers--working in 

similar situations on new facilities can apply the knowledge. A lesson 

learned is thus the product of a process through which lessons are 

captured and shared with stakeholders.



After a chemical agent was accidentally released at one of the 

project’s facilities in May 2000, some Members of Congress and state 

and local communities near disposal sites became increasingly concerned 

about the overall safety at the Chem-Demil Program’s incineration 

facilities. In July 2000, you requested that we report on the status of 

the Chemical Stockpile Emergency Preparedness Program and on the 

Programmatic Lessons Learned Program. We issued a report in August 2001 

on the Emergency Preparedness Program.[Footnote 1] For this second 

report, we (1) assessed whether the Lessons Learned Program has 

effectively captured and shared lessons to support the Chem-Demil 

Program’s goal to safely destroy the chemical stockpile and (2) 

identified the extent to which lessons learned have been shared and 

areas where sharing could be improved. You also asked us to provide 

additional information on incidents at three sites and the corrective 

actions taken following the incidents. The information is in appendix 

I.



In performing our analysis, we used the underlying principles of 

“knowledge management” and lessons sharing best practices as the 

criteria for assessing the program systems that capture and share 

lessons learned. Both DOD and the Army endorse lessons learned 

systems.[Footnote 2] Knowledge management includes four fundamental 

principles: leadership that articulates management’s vision and goals 

(e.g. in written policies and guidance), processes (including 

performance measurements) to turn vision into reality, technology that 

allows implementation of goals and supports the processes, and a 

culture of knowledge sharing and reuse. Together they create an 

environment in which a lessons learned program can successfully 

function.



We conducted our review from October 2001 to May 2002 in accordance 

with generally accepted government auditing standards. See appendix II 

for a description of our scope and methodology.



Results in Brief:



The Lessons Learned Program has successfully supported the incineration 

project’s primary goal to safely destroy chemical weapons. The program 

has captured and shared many lessons from past experiences and 

incidents. It has leadership that communicates the importance of the 

lessons learned program in supporting the Chem-Demil Program’s mission, 

processes for capturing and sharing lessons, and a technology to 

facilitate and support the program. It also has developed a culture 

that promotes using lessons to foster safe operations. However, the 

Lessons Learned Program does not fully apply generally accepted 

knowledge management principles and lessons sharing best practices, 

thereby limiting its effectiveness.



* The program’s management plan does not provide policy guidance for 

senior managers to help them in decision making or daily operations. 

Guidance is needed especially if managers decide not to implement a 

lesson learned. In at least one case, this resulted in cost avoidance 

prevailing with serious safety, cost, and schedule consequences. The 

program also does not define performance measures or provide incentives 

for participation.



* The Lessons Learned Program does not have formal procedures to test 

or validate whether a corrective action has been effective in resolving 

a deficiency.



* The lessons learned database is difficult to search and does not 

prioritize lessons. These shortcomings not only make it difficult to 

verify or validate corrective actions but also may discourage some from 

using the database, with potentially serious consequences.



The Lessons Learned Program has been effective in sharing knowledge 

among the different stakeholders within the Chemical Stockpile Disposal 

Project. However, as new components were created to destroy the 

stockpile, the scope of the Lessons Learned Program remained primarily 

limited to the incineration project. No policies or procedures were 

established to ensure that lessons sharing would expand to all 

components of the Chem-Demil Program. As a result, some components that 

could greatly benefit from timely and full sharing of lessons learned 

with the incineration project are not doing so. This can lead to higher 

risk and costly duplication and delays. The Assembled Chemical Weapons 

Assessment Program and the Alternative Technology and Approaches 

Project in particular could find full participation in the program 

useful because the majority of the processes they use are the same as 

those used by the incineration project. Lessons sharing best practices 

would dictate that all Chem-Demil Program components share important 

information such as lessons learned because they are all part of the 

same program with a common objective.



We are making recommendations to help improve the operation and overall 

usefulness of the Programmatic Lessons Learned Program.



Background:



In 1985, Congress required the Department of the Defense to destroy the 

U.S. stockpile of chemical agents and munitions and to establish an 

organization within the Army to manage the agent destruction program. 

Later, Congress also directed DOD to research and develop technological 

alternatives to incineration for disposing of chemical agents and 

munitions. These activities evolved into the Chem-Demil Program. The 

Chem-Demil Program includes the Chemical Stockpile Emergency 

Preparedness Program, created in 1988, to enhance the emergency 

management and response capabilities of communities near the storage 

sites in case of an accident. [Footnote 3] The Nonstockpile Chemical 

Materiel Product was added in 1993 to destroy any chemical weapons or 

materiel not included in the stockpile disposal program.



The Chemical Stockpile Disposal Project has or plans to use 

incineration to destroy chemical agents at five sites: Johnston Atoll 

in the Pacific Ocean; Anniston, Alabama; Pine Bluff, Arkansas; 

Umatilla, Oregon; and Tooele, Utah. Tooele is the only site with a 

facility currently operating. The three other stateside facilities are 

scheduled to begin operations in fiscal years 2002-2003. The Johnston 

Atoll facility has finished destroying its stockpile and is being 

closed. The Alternative Technologies and Approaches Project will use 

non-incineration methods (such as agent neutralization by chemical 

treatment) to destroy agents in bulk containers at Newport, Indiana, 

and Aberdeen, Maryland.[Footnote 4] The Assembled Chemical Weapons 

Assessment Program is also researching alternative methods to destroy 

agents in weapons at Pueblo, Colorado, and Blue Grass, Kentucky.



The Office of the Secretary of Defense and the Department of the Army 

share management roles and responsibilities in the Chem-Demil Program. 

The Program Manager of the Assembled Chemical Weapons Assessment 

Program reports to the Under Secretary of Defense for Acquisition, 

Technology, and Logistics. Thus, it is independent of the Program 

Manager for Chemical Demilitarization, who reports to the Assistant 

Secretary of the Army (Installations and Environment).



In 1997, the United States ratified the Chemical Weapons 

Convention,[Footnote 5] a treaty committing member nations to dispose 

of selected chemical agents and materiel by April 29, 2007. In 

September 2001, the Army updated the life cycle cost estimate for the 

Chem-Demil Program from $15 billion to $24 billion. The new cost 

estimate extended the agent destruction schedule at four of the eight 

stateside sites beyond the initial target date of April 2007.[Footnote 

6] Despite setbacks experienced at Johnston Atoll, Tooele, Utah, and 

Umatilla, Oregon, among others, the incineration program has 

successfully destroyed over 25 percent of the original stockpile (see 

table 1).



Table 1: Status of the Chemical Stockpile Disposal Project:



Percent of total stockpile destroyed; Johnston Atoll: ; 6; Tooele, 

Utah: ; 19; Anniston, 

Alabama: ; 0; Umatilla, 

Oregon: ; 0; Pine Bluff,

Arkansas: ; 0.



Start of operations; Johnston Atoll: June 1990; Tooele, 

Utah: Aug. 1996; Anniston, 

Alabama: 4th quarter FY02; Umatilla, 

Oregon: 4th quarter FY03; Pine Bluff,

Arkansas: 4th quarter FY03.



End of operations; Johnston Atoll: Nov. 2000; Tooele, 

Utah: 4th quarter FY05; Anniston, 

Alabama: 3rd quarter FY09; Umatilla, 

Oregon: 2nd quarter FY09; Pine Bluff,

Arkansas: 3rd quarter FY09.



Current phase; Johnston Atoll: Closure; Tooele, 

Utah: Operations; Anniston, 

Alabama: Systemization[A]; Umatilla, 

Oregon: Systemization[A]; Pine Bluff,

Arkansas: Construction.



[LEGEND] :



FY = fiscal year:



[A] Testing of each incineration system.



Source: Program Manager for Chemical Demilitarization.



[End of table]



The Lessons Learned Program was created in part because many different 

contractors were involved in the incineration program, and a system was 

needed to collect and preserve the institutional knowledge and acquired 

experience.[Footnote 7] The program is intended to identify, capture, 

evaluate, store, and share (implement) lessons learned during the 

different phases of the chemical stockpile demilitarization process. It 

collects two different kinds of lessons: “design” lessons covering 

engineering and technical processes and “programmatic” lessons 

involving management, quality assurance, emergency response, and public 

outreach. As criteria for assessing the knowledge management processes 

used by the Lessons Learned Program, we identified four of a number of 

federal organizations that practice knowledge management and operate 

lessons learned programs. In making our selections, we reviewed 

literature and spoke with knowledge management experts to find 

organizations recognized for their ability to share lessons or 

effectively manage knowledge. We identified the following 

organizations: the Center for Army Lessons Learned, the Department of 

Energy, the U.S. Army Corps of Engineers, and the Federal Transit 

Authority (for more details, see appendix II).



There are two levels of authority involved in developing lessons 

learned from proposed engineering changes. A Configuration Control 

Board composed of headquarters staff in the Office of the Program 

Manager for Chemical Demilitarization has authority to approve, reject, 

or defer engineering change proposals that involve costs above a set 

limit or affecting multiple sites. The Field Configuration Control 

Boards have authority over changes at their sites involving lower 

costs. In September 2001, the Lessons Review Team (consisting of 

headquarters staff) was established to screen all lessons and 

engineering changes and provide the information needed to determine 

which lessons require a response from sites. For more information on 

the lessons learned process, see appendix III.



Lessons Learned Program Has Made Positive Contributions but Needs 

Improvement:



The Lessons Learned Program has made valuable contributions in support 

of the Chemical Stockpile Disposal Project’s efforts to safely destroy 

the chemical stockpile. It has generally operated consistently with 

knowledge management principles and lessons sharing best practices and 

has successfully captured and shared thousands of lessons. However, the 

program does not apply or incorporate all knowledge management 

principles and lessons sharing best practices. For example, the program 

does not provide needed guidance for senior managers; it does not have 

formal a validation procedure to determine whether a problem has been 

fixed; and the database of lessons learned needs improvement.



Important Program Contributions:



The Lessons Learned Program has contributed to the Chem-Demil Program’s 

goal of destroying the chemical weapons stockpile while promoting 

safety, maintaining schedule, and saving or avoiding costs. We found 

that the Chem-Demil Program’s management, through its leadership, 

encourages headquarters, field staff, and contractor personnel in the 

incineration program to use the Lessons Learned Program. It has 

provided funding and has established processes to capture, evaluate, 

store, and share lessons. It is committed to continuous improvement and 

has provided the technology needed to support the lessons learned 

process. Finally, it fosters a culture in which knowledge sharing is an 

important element of day-to-day operations.



While it is difficult to quantify the benefits of each lesson, 

available data indicate that lessons learned have generally helped 

avoid on-the-job injuries (by using government-furnished-approved 

tools that are better suited to specific tasks), reduce costs (by 

improving the containers used to transport weapons), or maintain 

schedules (by improving the design of a socket to disassemble weapons). 

We also found that lessons from accidental releases of chemical agents 

at Johnston Atoll and Tooele, Utah, were implemented at other 

incineration sites under construction, thus incorporating improvements 

into the design of those new facilities.



Program Lacks Guidance to Support Managers’ Decision Making:



The Lessons Learned Program does not have guidance explaining how 

senior managers (at headquarters) should use it in support of their 

decision making process. Specifically, there is no guidance that 

defines the procedures to be followed when an alternative to a lesson 

is chosen or when a lesson is not implemented. Lessons learned guidance 

for another federal government agency recommends that lessons be used 

to optimize management decision making and to interact with other 

management tools such as reviews, investigations, root-cause analyses, 

and priorities.[Footnote 8]



We reviewed documentation of lessons learned from incidents at the 

Johnston Atoll and Tooele, Utah facilities, and found that three other 

facilities--Anniston, Umatilla, and Pine Bluff--had not implemented a 

lesson that had evolved from problems with pipes in the pollution 

abatement systems.[Footnote 9] The Tooele site had used a superior and 

more expensive material (hastelloy) to fix their problem than the 

material used at the other sites.[Footnote 10] Headquarters decided not 

to implement the lesson at the three sites primarily because it would 

have involved higher initial costs.[Footnote 11] This decision 

ultimately caused serious safety concerns, higher costs, and delayed 

the schedule. In February 2002, pipes at Anniston had failures similar 

to those experienced at the first two sites. This raised safety 

concerns and resulted in a 4-week delay to replace the pipes with 

hastelloy. It is too early to determine whether the material used at 

the Umatilla and Pine Bluff sites will have the same problems. Although 

they need flexibility to manage the program, senior managers also need 

guidance to help make decisions that allow them to consider the 

potential impact of not implementing lessons learned. This process 

would include safety and risk analyses that can provide criteria should 

they decide not to adopt a lesson learned.



Program Lacks a Procedure to Validate Lessons Implemented:



There is no formal procedure to ensure that the lessons or corrective 

actions that have been implemented have fully addressed a deficiency. 

Chem-Demil Program guidance for engineering change proposals does 

require that changes be tracked and reported after implementation, but 

there is no similar requirement in the guidance for the Lessons Learned 

Program (which includes programmatic lessons). Both contractor and 

incineration project officials also confirmed that there are no 

procedures for monitoring the effectiveness of corrective actions. As a 

result, a problem could reoccur and affect safety and costs.



As shown in figure 1, the Lessons Learned Program process does not 

contain the final validation stage (dashed line), which most knowledge 

management systems and Army guidance consider as a necessary step. As 

we previously reported, Army guidance states that lessons learned 

programs should have a means for testing or validating whether a 

corrective action has resolved a deficiency.[Footnote 12] The standard 

issued for another federal lessons learned program[Footnote 13] 

indicates that analyses should be made to evaluate improvements or to 

identify positive or negative trends. The standard also states that 

corrective actions associated with lessons learned should be evaluated 

for effect and prioritized. Without such a validation procedure in the 

architecture of the Lessons Learned Program, there is little assurance 

that problems have been resolved, and the possibility of repeating past 

mistakes remains.



Figure 1: Chem-Demil Programmatic Lessons Learned Program Process:



[See PDF for image]



Note: PLL (Programmatic Lessons Learned) is referred to in this report 

as the Lessons Learned Program; in the figure, engineering change 

proposal is referred to as ECP.



Source: Our analysis, based on data from PLL.



[End of figure]



Database Is Difficult to Use, Lessons Are Not Prioritized:



The lessons learned database includes about 3,400 issues, 3,055 

engineering change proposals, and 2,198 lessons. But it is not easy to 

obtain fast and ready access to relevant information. Furthermore, the 

lessons in the database are not prioritized, making it difficult to 

identify which lessons are most important and which need to be verified 

and validated.



It is important that an organization employ appropriate technology to 

support the participants of a lessons learned program. Having a 

technology be available does not automatically guarantee its use or 

acceptance. According to lessons sharing best practices,[Footnote 14] 

the goal of technology is to (1) match a solution to users’ needs, (2) 

establish a simple content structure so that items may be found easily 

and retrieved quickly, and (3) deliver only relevant information from 

all possible sources. According to database users we interviewed and 

surveyed, it is difficult to find lessons because the search tool 

requires very specific key words or phrases, involves multiple menus, 

and does not link lessons to specific events. As a result, some users 

are reluctant to use the database and thus may not benefit from it when 

making decisions that affect the program.[Footnote 15] Many users who 

responded to our survey stated that they experienced difficulties in 

searching the database, and some we interviewed described specific 

problems with searches. One described the database as “frustrating.” We 

tested the search tool and also had difficulty finding lessons linked 

to specific incidents.



Users we interviewed made a number of suggestions to improve the 

Lessons Learned Program’s database, including:



* improving the search capability,



* organizing by subject matter,



* ranking or prioritizing lessons,



* creating links to other documents,



* providing a Web-based link to the database,



* periodically purging redundant data, and:



* making access screens more user-friendly.



Furthermore, because the database does not prioritize lessons, managers 

may be unaware of some important areas or issues that need to be 

monitored or lessons that need to be reviewed and validated. By 

contrast, lessons learned processes used by the selected federal 

agencies include periodic reviews of the usefulness of lessons and the 

archiving of information that is no longer pertinent or necessary. The 

processes also include prioritizing lessons by risk, immediacy, and 

urgency. In 1998, the Army Audit Agency recommended that the database 

be purged or archived of obsolete items and that current and future 

lessons be prioritized. In September 2001, the Chem-Demil Program 

created a Lessons Review Team to begin identifying “critical” lessons 

(those requiring a response). But the team is not prioritizing lessons.



Some Knowledge Management Principles Are Not Applied:



Several other areas also did not adhere to knowledge management 

principles and lessons sharing best practices. For example, the Chem-

Demil Program’s management plan does not explain how the Lessons 

Learned Program is to achieve its goals or define performance measures 

to assess effectiveness. Knowledge management principles stress the 

importance of leaders articulating how knowledge sharing will be used 

to support organizational goals. Furthermore, the Chem-Demil Program 

does not provide incentives to encourage involvement in the Lessons 

Learned Program. Lessons sharing best practices and knowledge 

management principles prescribe developing and using performance 

measures to determine the effectiveness of a program. In addition, the 

Lessons Learned Program currently surveys employees after workshops to 

measure their satisfaction; however, these surveys are not sufficient 

to assess the overall effectiveness of the program. The program is 

attempting to identify ways to measure the cost and benefits derived 

from lessons learned. Knowledge management principles also encourage 

using performance evaluation, compensation, awards, and recognition as 

incentives for participation in lessons learned programs. The lack of 

incentives in the Lessons Learned Program may lead to missed 

opportunities for the identification and sharing of lessons learned.



Sharing of Lessons Learned Could Be Expanded:



The Lessons Learned Program has shared thousands of lessons among the 

five incineration sites through the different phases of construction, 

testing, and destruction of chemical agents. However, as the Chem-Demil 

Program evolved through the 1990s, and as the components using 

alternative technologies were added, the scope of the Lessons Learned 

Program did not expand to share lessons with the new components (see 

app. IV for a history of the Chem-Demil Program’s evolution). The 

Lessons Learned Program remained primarily focused on the five 

incineration sites. At the same time, each stockpile destruction 

component developed its own separate lessons learned, but without any 

program wide policies or procedures in place to ensure coordination or 

sharing of information across components.[Footnote 16] We reported in 

May 2000 that effective management of the Chem-Demil Program was being 

hindered by a complex organizational structure and ineffective 

coordination.[Footnote 17] This has created barriers to sharing.



Today, the four sites that are likely to use alternative technologies 

are not full participants in the lessons learned effort:



* The Assembled Chemical Weapons Assessment Program does not fully 

participate in the lessons learned process or activities. In at least 

one instance, the Assembled Chemical Weapons Assessment Program 

requested (from the Program Manager for Chemical Demilitarization), a 

package of data including lessons on the pollution abatement system 

filters, mustard thaw, and cost estimates. The data were eventually 

provided, but they were too late to be used during a DOD cost data 

review.[Footnote 18] This lack of access forced the program to submit 

incomplete cost data for the review because it was unable to obtain 

information from the incineration project in a timely manner.



* The Alternative Technologies and Approaches Project does have access 

to the Lessons Learned Program’s database, and it plans to develop its 

own separate database that it will share with the Lessons Learned 

Program only at “key milestones.” The project’s information, however, 

could be very valuable to other components of the Chem-Demil Program, 

especially the Assembled Chemical Weapons Program, which also 

researches alternative technologies. This plan could lead to lost 

opportunities and duplication of efforts.



Many of the lessons learned by the incineration project could be used 

by the other components of the Chem-Demil Program to promote safe, 

cost-effective, and on-time operations. Many of the technical processes 

(storing, transporting, unloading, and disassembling weapons) and 

programmatic processes (regulatory compliance, management, public 

relations practices) used by the Chemical Stockpile Disposal Project 

are very similar to those used by the other programs. This is also the 

case for processes used to develop operating destruction, or 

throughput, rates and cost and schedule projections. In fact, the 

majority of processes at incineration facilities are the same as those 

used by the Assembled Chemical Weapons Assessment Program and the 

Alternative Technologies and Approaches Project. Under these 

circumstances, promoting a culture of knowledge sharing would enable 

all components to capture and use organizational knowledge.



Furthermore, there is the possibility that the Pueblo, Colorado, site 

(and possibly the Blue Grass, Kentucky, site) now managed by the 

Assembled Chemical Weapons Assessment Program, which now reports to a 

DOD office, may be transferred to the Army’s Chem-Demil Program. If 

this transfer of responsibilities does take place, it would be 

important for the two programs to be already sharing information fully 

and seamlessly. Even if the transfer does not take place, knowledge 

management principles and lessons sharing best practices both dictate 

that components of the same program should share information, 

especially if they all have a common goal.



Conclusions:



The Lessons Learned Program has made important contributions to the 

safe destruction of the nation’s stockpile of chemical weapons. We 

found that the program generally adheres to knowledge management 

principles and lessons sharing best practices. However, the program’s 

full potential has not been realized. The program needs guidance to 

help senior managers make decisions that allow them to weigh the 

potential impact of not implementing lessons learned. This guidance 

would be a set of procedures, including safety and risk analyses, to be 

followed before deciding to counter a lesson learned. Without such 

guidance, decision makers, in at least one case, chose lower cost over 

safety and schedule, ultimately at the expense of all three. Also, the 

Lessons Learned Program lacks procedures to validate the effectiveness 

of implemented lessons. The lack of a validation step partially defeats 

the purpose of the lessons learned process, which relies on the 

confirmed effectiveness of solutions emerging from knowledge and 

experience. If the effectiveness of a lesson cannot be validated over 

time, problems may emerge again, with a negative impact on safety, 

costs, and schedule.



Further, the information in the lessons learned database is not easily 

accessible or prioritized. These drawbacks have frustrated users and 

may discourage them from using the database. This could lead to wrong 

or misinformed decisions that could affect safety. In addition, there 

is no overarching coordination or sharing of information across all the 

components of the Chem-Demil Program, which grew and evolved over time 

without policies or procedures to ensure that knowledge would be 

captured and communicated fully. As a result, fragmented or duplicative 

efforts continue today, and the Assembled Chemical Weapons Assessment 

Program in particular lacks access to important data maintained by the 

Chemical Stockpile Disposal Project and the Alternative Technologies 

and Approaches Project. In the case of the Chem-Demil Program, the 

absence of policies and procedures promoting and facilitating the 

broadest dissemination of lessons learned places the safety, cost 

effectiveness, and schedule of the chemical weapons destruction at 

risk.



Recommendations for Executive Action:



To improve the effectiveness and usefulness of the Chemical 

Demilitarization Program’s Lessons Learned Program, we recommend that 

the Secretary of Defense direct the Secretary of the Army to:



* develop guidance to assist managers in their decision making when 

making exceptions to lessons learned,



* develop procedures to validate, monitor, and prioritize the lessons 

learned to ensure corrective actions fully address deficiencies 

identified as the most significant, and:



* improve the organizational structure of the database so that users 

may easily find information and develop criteria to prioritize lessons 

in the database.



We also recommend that the Secretary of Defense direct the Secretary of 

the Army to develop policies and procedures for capturing and sharing 

lessons on an ongoing basis with the Alternative Technology and 

Approaches Project and in consultation with the Under Secretary of 

Defense (Acquisition, Technology, and Logistics) develop policies and 

procedures for capturing and sharing lessons on an ongoing basis with 

the Assembled Chemical Weapons Assessment Program.



Agency Comments and Our Evaluation:



The Army concurred with our five recommendations and provided 

explanatory comments for each one. However, these comments do not 

address the full intent of our recommendations. With regard to our 

recommendation that it provide guidance to assist managers when 

deciding to make an exception to a lesson, the Army stated that the 

Lessons Review Team has guidance for characterizing the severity level 

of lessons learned. However, as our report clearly points out, this 

guidance is for site officials and is insufficient in assisting senior 

managers at headquarters on important decisions involving costly 

lessons that could potentially impact several sites. We believe that 

good management practices require that senior managers make decisions 

based on risk, safety, and cost analyses and that guidance should be 

developed to support this decision-making process as we recommended.



In concurring with our recommendation to develop procedures to ensure 

corrective actions fully address deficiencies, the Army stated that it 

is initiating an effort whereby the system’s contractors will be 

responsible for validating, monitoring, and prioritizing lessons. The 

Army’s Lessons Learned Program currently does not validate the results 

of corrective actions. Contracting this important function will require 

monitoring by the Chem-Demil program to ensure that validation is 

properly conducted as we recommended.



The Army stated that it has improved the Lessons Learned database to 

make it easier to locate information. Converting the database to an 

Internet-based program should also improve its accessibility and 

utility. Although these actions address some users’ concerns, the Army 

needs to address all related user issues identified in our report in 

order to improve the benefits of the database.



The Army concurred with our recommendation to develop policies and 

procedures to capture and share lessons with the two alternative 

technology programs. It stated that progress had been made toward 

sharing lessons between the Alternative Technologies and Approaches 

Project and the Lessons Learned Program at key milestones. The Army 

also said it has shared the lessons database with the Assembled 

Chemical Weapons Assessment Program. However, the Army should require, 

as we recommended, that policies and procedures for capturing and 

sharing lessons on an ongoing basis be established, instead of sharing 

at key milestones and on a one-way basis. This approach would ensure 

that both alternative technology programs fully participate in the 

Lessons Learned Program and that the database is constantly enriched to 

enhance safety, cost, and schedule based decisions for all components 

of the Chem-Demil program.



The Army’s comments are printed in appendix V. The Army also provided 

technical comments, which we incorporated where appropriate.



We are sending copies of this report to interested congressional 

committees, the Secretaries of Defense and of the Army; the Assistant 

Secretary of the Army (Installations and Environment); the Under 

Secretary of Defense (Acquisitions, Logistics, and Technology); the 

Director, Federal Emergency Management Agency; and the Director, Office 

of Management and Budget. We will also make copies available to others 

upon request. In addition, the report will be available at no charge on 

the GAO Web site at http://www.gao.gov:



Please contact me at (202) 512-6020 if you or your staff have any 

questions regarding this report. Key contributors to this report were 

Donald Snyder, Bonita Oden, Pamela Valentine, Steve Boyles, and Stefano 

Petrucci.



Raymond J. Decker

Director, Defense Capabilities

and Management:



Signed by Raymond J. Decker:



[End of section]



Appendix I: Information on the Incineration Process and Incidents at 

Three Sites:



There have been three releases of agent from operating incineration 

facilities and one incident during construction that have generated 

several lessons learned. The incineration process and the releases and 

construction incidents are described below.



The Army’s Baseline Incineration Process:



A baseline incineration process uses a reverse-assembly procedure that 

drains the chemical agent from the weapons and containers and takes 

apart the weapons in the reverse order of assembly. Once disassembled, 

the chemical agent and weapon parts are incinerated in separate 

furnaces and the gaseous and solid waste is treated in a separate 

process. Liquid brine resulting from the treatment of exhaust gases in 

the pollution abatement system is dried to reduce the volume and 

transported to a commercial hazardous waste management facility.



The path to weapons disposal, in general includes six major steps.



Chemical weapons are stored in earth-covered, concrete-and steel 

buildings called igloos. These igloos are guarded and monitored for any 

signs of leaking weapons by the U.S. Soldier and Biological Chemical 

Command.



Chemical weapons are taken from the igloos and transported to a 

disposal plant in sealed on-site containers by the U.S. Soldier and 

Biological Chemical Command. The sealed containers are resistant to 

fire and impact.



When the on-site containers arrive at the disposal plant, workers check 

them for leaking weapons before opening them. Chem-Demil crews then 

load the weapons onto conveyors that carry the weapons through the 

disposal process. When the weapons are loaded onto the conveyor, the 

U.S. Soldier and Biological Chemical Command no longer has 

responsibility for them.



From this point on, workers manage the disposal process from an 

enclosed control room using advanced robotics, computer technology, and 

video monitoring equipment. Automatic, robotic equipment drains the 

chemical agent from the weapon and takes the weapons apart in explosive 

proof rooms.



Once dismantled and drained, the individual weapon parts travel to 

different furnaces in the plant, each designed for a specific purpose. 

The liquid incinerator destroys the chemical agent, the deactivation 

furnace destroys explosive materials, and the metal parts furnace heats 

shell casings and other heavy metal parts to destroy any remaining 

agent contamination.



The pollution abatement system cleans the air before it is released 

into the environment.



Agent Release at Tooele:



The Tooele Chemical Agent Disposal Facility (Tooele plant) is located 

on Deseret Chemical Depot in Tooele, Utah. The facility is designed to 

dispose of 44.5 percent of the nation’s original stockpile of chemical 

weapons. Tooele plant is the first chemical weapons disposal facility 

built within the continental United States. Construction of the Tooele 

plant began in October 1989 and disposal operations began in August 

1996. Operations at Tooele plant should be completed in 2008. The 

Tooele plant incorporates systems originally tested and used at the 

Chemical Agent Munitions Disposal System, also located at the depot. 

These systems were first used on an industrial scale at the Army’s 

Johnston Atoll Chemical Agent Disposal System (Johnston Atoll plant) in 

the Pacific Ocean. The Johnston Atoll plant was the first integrated 

facility built to dispose of chemical weapons.



The sequence of events described in table 3 is based on documents from 

the Utah Department of Environmental Quality--Division of Solid and 

Hazardous Waste, U.S. Army Safety Center, Department of Health and 

Human Services--Centers for Disease Control and Prevention, and a 

program contractor. On May 8, 2000, the day shift was processing 

rockets in the deactivation furnace system. The deactivation furnace 

system lower tipping gate (used to control the feed of munitions to the 

furnace) did not close properly and munitions/agent processing was 

terminated. Workers in protective gear began to clean and repair the 

gate and a strainer. A bag from the strainer, contaminated with GB 

(nerve) agent, was left on top of the gate. This is believed to be the 

source of the agent that was released. Vapors were drawn from the bag 

through the furnace system.



During the initial attempt to re-light the afterburners following the 

cleaning procedure, the agent monitoring equipment alarmed. During a 

second attempt to re-light these burners another agent monitor alarmed. 

In summary, a small amount of agent escaped through the common stack 

during attempts to relight the furnace. (See table 2.):



The several corrective actions taken were based on 105 investigation 

findings involving operations, training, and equipment. Lessons learned 

from this incident include (1) modifying feed chute clean out 

procedures, (2) providing operator refresher training, (3) installing a 

deactivation furnace remote operated valve to isolate the deactivation 

furnace during afterburner re-lights, and (4) redesigning deactivation 

furnace feed chute.



Table 2: May 2000 Agent Release at Tooele Chemical Agent Disposal 

Facility:



Date/time: May 8, 2000; Event description: Team C, working the day 

shift, was processing M56 warheads in the deactivation furnace system 

(DFS) and spent decontamination solution (SDS) in the liquid 

incinerator (LIC) #1..



Date/time: 4:00 P.M.; Event description: The lower tipping feed gate 

(from the explosive containment room) on the deactivation furnace 

system was sticking.; Operators began preparation for a two-man entry 

(in demilitarization protective ensemble level dress) to clear the jam 

in the lower tipping feed gate..



Date/time: 6:00 P.M.; Event description: Team A relieved Team C, and 

the problem with the lower tipping valve was briefed to the oncoming 

shift.; The DFS chute sprays were on at the time of the operator 

change..



Date/time: 8:10 P.M.; Event description: The pressure in the DFS rotary 

kiln was lowered. This lower pressure in the kiln increased the airflow 

rate through the system. The major problem was that the pressure began 

to oscillate significantly.; This reduction lowered the time agent--

produced gases were exposed to heat in the DFS afterburner..



Date/time: 8:20 P.M.; Event description: A DFS Afterburner Exhaust Flow 

Sensor alarm occurred indicating low pressure and high air through the 

DFS incinerator and the pollution abatement system.; Operator has 

trouble controlling pressure..



Date/time: 8:37 -9:30 P.M.; Event description: The entrants prepared to 

use water to power-wash the debris that caused the tipping feed gate-

sticking problem. This water hose malfunctioned.; The entrants left the 

explosive containment room, repaired the hose, and returned to 

completed the clean up.; Once in the explosive containment room, the 

entrants attempted to use a droplight to get a better view, the 

droplight did not work. The entrants left, retrieved a working 

droplight, and returned for a third time to the explosive containment 

room to complete the tipping feed gate maintenance.; The entrants had 

to leave the explosive containment room again, this time to repair a 

clamp on the water hose.; The entrants cleaned out the Agent 

Quantification System strainer and placed the strainer sock on the 

upper feed gate. The sock contained about one pound of agent-

contaminated fiberglass fragments..



Date/time: 8:42 P.M.; Event description: The DFS Operator noticed 

pressure fluctuations that began to affect the DFS induced draft fans. 

(These fans pull air through the DFS incinerator and pollution 

abatement system.).



Date/time: 8:48 P.M.; Event description: The DFS operator took manual 

control of the kiln pressure controller and venturi plug valve.; The 

wash down of the chute was completed by 9:30 p.m..



Date/time: 9:45 P.M.; Event description: The DFS operator has a 

difficult time stabilizing the DFS..



Date/time: 9:59 P.M.; Event description: The DFS exhaust flow sensor 

sends a malfunction signal to the control room, the flow sensor/meter 

had been saturated with liquid during the entrants’ maintenance 

operation on the tipping feed gate.; This was followed by an alarm that 

automatically shuts down the burner in the DFS kiln and in the DFS 

afterburner.; Large draft pressure moved water into the meter.; The 

temperatures in both burners dropped below permit levels.; The DFS 

operators are unaware of a major agent source presence (the strainer 

sock on the upper feed gate left by the entrants)..



Date/time: 10:26 P.M.; Event description: The DFS operators began 

attempts to re-light the burners; they felt that re-lighting the 

burners would be the safest course of action for preventing a release 

of agent.; The DFS operators increased the combustion air in an attempt 

to re-light the afterburner; by 10:48 p.m. a decision was made to stop 

trying to re-light the burners..



Date/time: 11:18 P.M.; Event description: The operators shut down the 

clean liquid pump. This was done to assist in drying out the flow 

sensor/meter..



Date/time: 11:26 P.M.; Event description: The first stack agent monitor 

alarm occurred.; This was ignored because the duct alarm did not signal 

and it should have alarmed first.; At this time the site was masked 

(workers were instructed to use protective masks).; The temperature in 

the DFS kiln was approximately 204 F lower than what is required to 

destroy agent and the temperature in the DFS afterburner was 

approximately 1,250 F lower than the requirement..



Date/time: 11:27 P.M.; Event description: A second agent monitor alarm 

occurred..



Date/time: 11:30 P.M.; Event description: The control room operator 

notified the depot emergency operations center.; The depot commander at 

the emergency operations center did not make contact with the Tooele 

County emergency responders until nearly 4 hours after the first alarm 

at 3:34 a.m. on May 9, 2000..



Date/time: 11:38 P.M.; Event description: The Depot Area Air Monitoring 

System tubes for the common stack were removed for testing.; The 

analysis confirmed the presence of GB chemical agent..



Date/time: 11:41 P.M.; Event description: A third agent monitor alarmed 

occurred..



Date/time: 11:44 P.M.; Event description: The control room operator 

directed a “bottle-up” of the DFS, in essence closing dampers, slowing 

air flow in order to slow the loss of temperature to in the DFS.; 

Residence time in the DFS afterburner climbed and the afterburner 

temperature began to rise..



Date/time: May 9, 2000; 12:18 A.M.; Event description: Notice to unmask 

the site was given..



Date/time: 12:23 A.M.; Event description: The DFS operator attempted a 

second re-light of the DFS afterburner. A re-light was initiated..



Date/time: 12:28 A.M.; Event description: During the re-light, the 

common stack and DFS agent monitors alarmed again.; The site was 

masked..



Date/time: 12:32 A.M.; Event description: The DFS operator was directed 

to “bottle-up” the furnace again..



Date/time: 1:07 A.M.; Event description: The site was unmasked..



Date/time: 1:17 A.M.; Event description: The depot emergency operations 

center received notification that the Depot Area Air Monitoring System 

analysis confirmed the presence of agent..



[End of table]



Agent Releases at Johnston Atoll:



In addition to reviewing lessons from Tooele incidents, we were briefed 

on two incidents that occurred at Johnston Atoll, and we reviewed 

relevant investigation reports for these incidents. Both incidents 

resulted in corrective actions and generated several lessons learned.



On March 22, 1994, the liquid agent gun purge process began. The next 

day workers dressed in protective gear removed the liquid agent gun, 

and three lines had to be disconnected and capped (sealed). These three 

lines to the liquid agent gun are the atomizing air, fuel oil, and the 

agent line. During the disconnecting of the agent line, the liquid 

incineration room agent monitoring system alarmed. Also, the agent 

monitors in the common stack began to alarm. Operators turned off the 

induction fan to divert room air out through plant exhaust to the 

carbon filters.



Lessons learned from this incident include (1) replacing the fuel oil 

purge system flow meter with an instrument that could be read in the 

control room; an investigation found that the flow meter on the agent 

purge line was not functioning (2) directing room air away from the 

pollution abatement system to prevent contaminated air from escaping 

through the duct work without going through the furnace and (3) 

counseling workers on the importance of following approved standard 

operating procedures.



On December 8, 1990, a laboratory analysis confirmed emission of 

chemical agent from the common stack following a purging (flushing) of 

the agent line. It was determined that the probable cause of the 

release was that a quantity of agent GB (nerve) leaked from the agent 

gun or feed line into the primary chamber of the liquid incineration 

furnace, and the agent was swept downstream by the induced draft fan 

(used to draw air through the plant) while the furnace was in a cool-

down cycle. It appears that the agent that leaked into the incinerator 

and ultimately discharged to the atmosphere was from either valves in 

the agent feed line to the primary chamber that were not totally sealed 

or the agent remained in the agent line after it was purged and was 

aspirated into the incinerator and subsequently the atmosphere. During 

the incident, and due to a malfunctioning agent-sampling probe, the 

agent-monitoring equipment in the common stack did not detect agent.



Lessons learned from this incident include (1) improving the process to 

purge (flush) chemical agent from the feed line by adding a fuel oil 

purge and increasing the purge cycle to ensure a complete purge, (2) 

modifying the alarm system in the common stack to provide redundancy 

and test the alarms more frequently, and (3) closing all four valves 

after the agent line is purged and process activities involving the 

liquid incineration feed system when the furnace is cooling down to the 

charcoal filters.



Construction Incident at Umatilla:



On September 15, 1999, more than 30 construction workers were affected 

by an irritating vapor in the air while working in the munitions 

demilitarization building. This incident caused many workers to 

experience respiratory irritation, sending them to the local hospital 

where they were examined and released. Later that day, all construction 

work stopped and approximately 800 contracted workers were sent home. 

Investigations and analyses lead to the determination that chemical 

agent was not involved; instead this was determined to be a 

construction incident.



As construction progressed, the building became a “closed-in” area and 

may not have been adequately ventilated. The building ventilation 

system was not designed to control contaminants during construction; it 

was only intended to control a release of chemical agent when 

construction was complete and operations had begun. The release of 800 

contracted-construction workers without informing them of the situation 

that no chemical agents were involved, coupled with the slow release of 

information to the press, eventually heightened public concern.



Lessons learned from this incident include (1) enhancing local 

ventilation in the munitions demilitarization building, (2) 

establishing and posting evacuation routes and response procedures 

throughout the site, (3) installing a temporary public address system 

at the construction site, and (4) ensuring there is adequate 

communications between the site and any off-site facilities 

particularly in the event of an incident.



Agent Exposure at Tooele:



On July 15, 2002, at the time we were drafting this report, an 

individual working at the incineration facility in Tooele, Utah, 

experienced a confirmed accidental chemical agent exposure. This 

individual was performing maintenance on an agent purge line valve in 

the liquid incinerator room and was exposed to residual agent present 

in the agent purge line. The worker exhibited symptoms of chemical 

agent exposure. Although the Army, DOD Inspector General, and the 

facility’s contractor are conducting investigations looking into the 

events associated with the accidental exposure, it is too early to 

report on lessons resulting from this incident. The Program Manager for 

Chemical Demilitarization is awaiting the investigation reports and 

will incorporate the corrective actions into lessons learned. According 

to the Army, agent operations will not commence until all corrective 

actions have been taken and the plant is deemed safe to operate.



[End of section]



Appendix II: Scope and Methodology:



To assess the Lessons Learned Program, we reviewed literature on the 

principles of knowledge management and our previous reports on lessons 

sharing best practices.



* To assess the leadership of the Lessons Learned Program, we 

interviewed Chem-Demil Program managers, personnel, and the contractor 

staff who manage the Lessons Learned Program. We also reviewed 

management documents describing the program and we conducted 30 

structured interviews[Footnote 19] with the Chem-Demil Program’s 

managers (headquarters and field level) and systems contractor staff at 

three sites (Aberdeen, Maryland; Anniston, Alabama; and Tooele, Utah) 

to determine how clearly management articulated its expectations about 

using lessons learned. We did not select a statistical sample of 

database users; therefore, our survey results cannot be generalized to 

all Lessons Learned Program database users.



* To describe the lessons learned process, we reviewed documentation 

relevant to the lessons learned process. We also interviewed personnel 

from the office of the Program Manager for Chemical Demilitarization, 

the Anniston, Alabama, site, and the contractor responsible for 

managing the Lessons Learned Program.



* To learn how technology supports the Lessons Learned Program, we 

reviewed the lessons learned process and identified the methods used to 

gather, consolidate, and share information with stakeholders. We also 

asked the staff we surveyed how effectively does the program’s 

technology tools support the lessons learned process.



* To determine whether the Chem-Demil Program fosters a culture of 

knowledge sharing and use, we talked to program managers for each Chem-

Demil Program components, headquarters staff, and personnel from the 

lessons learned contractor staff to determine how lessons are shared 

and whether employees are encouraged to participate in the program. We 

also asked the staff we surveyed how frequently they submitted 

information to the program, whether they used the lessons, and whether 

there were incentives to encourage participation.



To determine whether lessons learned contributed to the goals of the 

destruction program; we documented and reviewed several important 

lessons that program staff identified. We also traced several lessons 

from incidents at Johnston Atoll and Tooele to verify that they had 

been shared and implemented at the Anniston facility. We used 

unverified Army data to assess whether the Lessons Learned Program 

achieved its aim of reducing or avoiding unnecessary costs. To 

determine if the Lessons Learned Program process conforms to other 

programs’ lessons sharing processes we identified four of a number of 

federal organizations that practice knowledge management and operate 

lessons learned programs. In making our selections, we reviewed 

literature and spoke with knowledge management experts to find 

organizations recognized for their ability to share lessons learned or 

effectively manage knowledge. We obtained information from the Center 

for Army Lessons Learned, the Department of Energy, the U.S. Army Corps 

of Engineers, and the Federal Transit Authority. We interviewed 

representatives from each organization about the processes they used 

for identifying, collecting, disseminating, implementing, and 

validating lessons learned information. We reviewed their lessons 

learned program guidance to compare and contrast their practices with 

the incineration project’s Lessons Learned Program process. We also 

interviewed an expert familiar with the program about the management of 

the lessons learned process. To assess the search, linkage, and 

prioritization of the database, we obtained documentation and 

interviewed the contractor staff about the information in the database. 

We tested the search feature of the database, including accessing 

menus, keyword and category listings, and analyzed several lessons 

learned we had obtained from our searches. We obtained opinions from 

the staff we surveyed on the effectiveness of the lessons learned 

database and their suggested areas of improvement. The respondents 

included managers and others with an average of 9 years experience in 

the Chem-Demil Program. The staff we surveyed routinely search the 

database for lessons learned information. We did not select a 

statistical sample of database users; therefore, our survey results 

cannot be generalized to all Lessons Learned Program database users.



To assess the extent to which lessons learned have been shared, we 

interviewed the Program Manager for Chemical Demilitarization and the 

contractor responsible for operating the Lessons Learned Program. We 

also attended status briefings for each Chem-Demil component. We 

focused our work primarily on the stockpile destruction projects/

programs. We conducted interviews with officials from the Alternative 

Technologies and Approaches Project, the Assembled Chemical Weapons 

Assessment Program, and the Chemical Stockpile Disposal Project to 

gather evidence on the commonality the alternative technology 

components have with the incineration program and the extent to which 

they share lessons learned information. To determine whether each 

component participated in the Lessons Learned Program by either sharing 

or receiving lessons learned information, we reviewed workshop minutes 

from calendar years 2000 and 2001.



To describe the incidents at three sites, we attended briefings on the 

incidents provided by officials from the incineration program, and 

reviewed incident investigation reports and entries in the Lessons 

Learned database. We identified key lessons from these sources and 

toured the Anniston Chemical Disposal Facility, to determine whether 

lessons learned had been shared and implemented. During our visit, we 

observed that several lessons from the Tooele incident, among others, 

were implemented.



[End of section]



Appendix III: Lessons Learned Process:



The Lessons Learned Program was established to collect and share 

lessons learned within the incineration program. The Programmatic 

Lessons Learned Program uses various methods to identify, review, 

document, and disseminate lessons learned information among government 

and contractor personnel. The program uses facilitated workshops to 

introduce lessons and also takes lessons from engineering change 

proposals. The Lessons Review Team reviews issues and determines 

specific lessons to be implemented. These issues, engineering changes, 

and lessons are stored in a database.



The program uses five distinct steps to develop lessons learned, as 

shown in figure 2.



* Issues are raised through topics submitted to workshops (meetings of 

headquarters and site personnel), critical document reviews (of changes 

to program documents), engineering change proposals (technical changes 

at one or more sites), quick reacts (immediate action), and express 

submittals (information from a site.):



* Experts review issues to determine if a change should be initiated in 

a workshop, an assessment (a study to support a management 

recommendation for change), engineering change proposal review process 

(a team at each site reviews changes at other sites), and directed 

actions (requests for information on actions a site has taken.):



* Lessons are identified from workshops, assessment reports, and the 

lessons review team (headquarters activity to segregate lessons into 

response required or not required.):



* Issues and lessons are stored in the database.



* Lessons are then shared with stakeholders, including contractor 

personnel, through access to the database, technical bulletins (a 

quarterly publication with information of general interest to multiple 

sites), programmatic planning documents (containing policies, 

guidelines, management approaches, and minimum requirements), and site 

document comparisons (new documents with baseline documents.):



Four primary elements of these steps are discussed below.



Figure 2: Lessons Learned Stakeholders and Process Steps:



[See PDF for image]



Note: PLL (Programmatic Lessons Learned) is referred to in this report 

as Lessons Learned Program; in the figure engineering change proposal 

is referred to as ECP.



Source: Project Manager for Chemical Stockpile Disposal, Programmatic 

Lessons Learned Program Plan, Revision 3, April 2002.



[End of figure]



Lessons Learned Program Facilitated Workshops:



Facilitated workshops are the primary method for introducing lessons 

learned into the Lessons Learned Program. Facilitated workshops are 

meetings that offer an environment conducive for site and headquarters 

personnel to speak openly about experiences. The intent of the 

workshops is to allow program personnel familiar with particular 

subjects to hold detailed discussions of issues relative to specific 

subjects. All issues discussed in the workshops are entered into the 

database and later reviewed to determine if the issues should become 

lessons learned.



The facilitated workshop process begins with a memorandum that requests 

site personnel from the Lessons Learned Program team to identify topics 

they want to discuss in workshops. These topics are generally divided 

into three basic categories: (1) valuable information provided to other 

sites, (2) challenging issues and discussion of issues with other sites 

in anticipation of possible recommendations, and (3) general topics to 

discuss different approaches to a problem. After each workshop, a 

feedback survey is sent to participants to determine user satisfaction 

with workshops.



Engineering Change Proposal Review Process:



Engineering change proposals are the primary method of approving and 

documenting design changes at the sites. Members of the Configuration 

Control Board and the Field Configuration Control Boards are 

responsible for reviewing and approving engineering change proposals 

within certain dollar limits. The Configuration Control Board, 

consisting of members from headquarters, is also responsible for 

managing changes to items or products identified for configuration 

control, such as facilities and equipment in order to maintain or 

enhance reliability, safety, standardization, performance, or 

operability. Each Field Configuration Control Board consists of members 

from a site, and is responsible for controlling engineering changes 

during construction, systemization, operations, and closure of 

facilities. Engineering change proposals are discussed during bi-weekly 

teleconferences where the sites can ask the originating site questions 

about the proposed engineering change.



The Field Configuration Board is responsible for approving engineering 

change proposals with an estimated cost of $200,000 or less. The 

Configuration Control Board is responsible for approving proposals with 

an estimated cost of $200,001 to $750,000. Proposals over $750,000 are 

sent to the Project Manager for Chemical Stockpile Disposal for 

approval. After approval, the engineering change proposals are reviewed 

and input into the database and sent to the Lesson Review Team as part 

of the review process.



Engineering changes are the primary source of design-related lessons 

learned. Engineering change proposals are approved changes in the 

design or performance of an item, a system or a facility. Such changes 

require change or revision to specifications, engineering drawing, and/

or supporting documents. Consequently, the Program Manager for Chemical 

Demilitarization developed a review process as a method to capture 

these lessons in the Lessons Learned Program. The purpose of the 

Engineering Change Proposal Review Process is to provide Chemical 

Demilitarization sites with more control over lessons learned decisions 

and incorporate lessons learned sharing under the Lessons Learned 

Program. Additionally, the review process is structured to allow each 

site the opportunity to review engineering changes being implemented at 

other sites and consider the applicability to their site. The review 

team consists of members from the sites, the Program Manager’s office, 

the Lessons Learned Program team, and the U.S. Army Corps of Engineers.



Lessons Review Team:



The Lessons Review Team, established in September 2001, is responsible 

for reviewing issues discussed in facilitated workshops to determine 

their potential impact and to determine if a specific site action is 

required. Additionally, the review team reviews engineering change 

proposals to determine if they are design-related lessons learned.



Issues are considered “lessons learned” when they have programmatic 

interest and significant impact on safety, environmental protection, or 

plant operations. The Lessons Review Team designates lessons learned as 

mandatory, “response required,” and “response not required.” A lesson 

is mandatory if the method of implementation has been or is directed 

from the Program Manager for Chemical Demilitarization headquarters. A 

lesson that is characterized as “response required” for means that the 

given site must provide information to the home office on the action 

taken to address the lesson. “Response not required” means that the 

site is not required to provide information to the headquarters on the 

action that the site has taken.



For mandatory lessons, the Lesson Review Team decision makers provide 

specific guidance for implementation of lessons. Technical support 

staff on the team conducts lesson reviews and provides recommendations 

to the decision maker regarding lessons. A team member is responsible 

for initial review of lessons and recommended designation, distribution 

of materials before the meetings, and facilitation of the meetings.



Lessons Learned Program Database:



The Lessons Learned Program database is a repository for (1) issues 

generated from facilitated workshops, (2) engineering change proposals, 

(3) critical document reviews, (4) quick react/advisory system and 

other lessons learned process data, and (5) programmatic and design 

lessons learned. As of April 2002, the database contained 3,400 issues, 

7,630 directed action, and 3,055 engineering change proposals.



The database was developed as a stand-alone program allowing users to 

employ search utilities or category trees to retrieve lessons. The 

program opens to the main screen, which consists of a search, 

categories, and lessons screens. The lessons screen is a search 

mechanism that utilizes a “drop down menu” enabling users to locate 

lessons by selecting categories or subcategories to narrow the search 

for lessons in a specific area. To summarize information and identify 

lessons in the lessons learned database, the database contains 

background information to support each lesson. The background 

information provides a condensed history, as well as the status of each 

lesson at the Chemical Demilitarization site.



[End of section]



Appendix IV: Chemical Demilitarization Program Management 
Developments, 

1997-2001:



The Departments of Defense and the Army made several changes to the 

management structure of the Chem-Demil Program, principally in response 

to congressional legislation. Originally the Program Manager for 

Chemical Demilitarization reported directly to the Assistant Secretary 

of the Army (Installations and Environment), who also oversees storage 

of the chemical weapons stockpile. The U.S. Soldier and Biological 

Chemical Command manages the stockpile. The Command also manages the 

loading, delivery, and unloading of chemical weapons at the destruction 

facility. After the estimated cost of the program reached a certain 

dollar amount, as required by statute,[Footnote 20] the Army formally 

designated it a major defense acquisition program. To manage this 

program in the Army acquisition chain, it was then transferred to the 

Assistant Secretary of the Army (Acquisition, Technology, and 

Logistics). The Program Manager for Chemical Demilitarization continued 

executing the program. In 1997, the Chemical Stockpile Emergency 

Preparedness Program was removed from the Program Manager for Chemical 

Demilitarization and transferred back to the Assistant Secretary of the 

Army (Installations and Environment) where it is currently managed by 

the U.S. Soldier and Biological Chemical Command. Also in 1997, the 

Army and the Federal Emergency Management Agency signed a new 

memorandum of agreement to better manage the on-and off-post emergency 

response activities, respectively.



In the 1997 Defense Appropriations Act (sec. 8065),[Footnote 21] 

Congress required the Assembled Chemical Weapons Assessment Program be 

independent of the Program Manager for Chemical Demilitarization and 

report directly to the Under Secretary of Defense (Acquisition and 

Technology).[Footnote 22] The purpose of this legislation was to 

separate the pilot program from the baseline incineration activities. 

Achievement of this goal also meant that two program offices would 

share responsibilities associated with disposal activities in Kentucky 

and Colorado. However, the pilot program’s legislation does not 

specifically state whether or not the Program Manager for Chemical 

Demilitarization will manage the assessment program once the 

development of technology evaluation criteria, the technology 

assessment, the demonstration, and pilot phases end.



In May 2000, we reported on the fragmented management structure and the 

inadequate coordination and communication within the Chem-Demil 

Program. [Footnote 23] We recommended that the Army should clarify the 

management roles and responsibility of program participants and 

establish procedures to improve coordination among the program’s 

various components.[Footnote 24] The Army, in December 2001, 

transferred the Chemical Demilitarization Program to the Assistant 

Secretary of the Army (Installations and Environment), bringing all 

components of the program, except the Assembled Chemical Weapons 

Program, under a single Army manager, as shown in figure 3. Another 

significant management change occurred in April 2002 when the Program 

Manager for Chemical Demilitarization retired after holding this 

position for the past 5 years.



Figure 3: Chemical Demilitarization Program Organization Chart:



[See PDF for image]



Source: Offices of the Chemical Demilitarization and Assembled Chemical 

Weapons Assessment Programs.



[End of figure]



[End of section]



Appendix V: Comments from the Department of the Army:



DEPARTMENT OF THE ARMY:



21 AUG 1002:



Mr. Raymond J. Decker Director:



Defense Capabilities and Management United States General Accounting 

Office Washington, D.C. 20548:



Dear Mr. Decker:



On July 23, 2002, the Office of the Inspector General of the Department 

of Defense forwarded a copy of the Draft GAO Report “CHEMICAL WEAPONS: 

Lessons Learned Program Generally Effective but Could Be Improved and 

Expanded (GAO-02-890)”, for review and comment on the report and the 

recommendations it contained. The Department of Defense has reviewed 

this report and concurs with the GAO findings and recommendations.



The enclosure details specific comments on the Draft report.



Sincerely,



Mario P. Fiori:



Signed by Mario P. Fiori:



Enclosure:



GAO DRAFT REPORT - DATED 19 JULY 2002 GAO-02-890:



“CHEMICAL WEAPONS: Lessons Learned Program Generally Effective but 

Could Be Improved and Expanded”:



DEPARTMENT OF THE ARMY COMMENTS TO THE RECOMMENDATIONS:



RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 

direct the Secretary of the Army to:



*develop guidance to assist managers in their decision making when 

making exceptions to lessons learned;



*develop procedures to validate, monitor, and prioritize the lessons 

learned to ensure corrective actions fully address deficiencies 

identified as the most significant; and:



*improve the organizational structure of the database so that users may 

easily find information and develop criteria to prioritize lessons in 

the database.



(p. 15/GAO Draft Report):



ARMY RESPONSE:



a. Develop Guidance to Assist Managers. Concur:



Guidance was developed in September-October 2001. The Project Manager 

for Chemical Stockpile Disposal (PMCSD) has chartered a Lessons Review 

Team to assist managers in their decision-making. This assistance is 

provided by first prioritizing the lessons (see response below for the 

severity level definitions), then by requiring a response from the site 

to PMCSD. A response from the site is a commitment to implement the 

lesson learned or to provide documentation of why the lesson learned 

should not or will not be implemented. Severity Level II lessons 

require engineering/managerial judgment to determine whether a response 

from the site will be required. Severity level III lessons do not 

require a site response and are assigned to the site to determine the 

disposition of the lesson. Exceptions made to lessons learned that are 

of programmatic interest are made after extensive review by the Lessons 

Review Team and the site involved.



b. Develop Procedures to Validate, Monitor and Prioritize the Lessons. 

Concur:



Under the Lessons Review Team charter, guidelines and criteria have 

been developed for those lessons that are of programmatic interest to 

prioritize each lesson. Severity levels are assigned to each lesson.



Severity Level I - Could impact containment of chemical agent or 

explosion; potentially expose workers to chemical agent; cause 

significant harm to workers due to industrial activities; result in 

release of toxic/hazardous material affecting public and/or worker 

health, safety or the environment; or result in extensive damage to 

equipment or long term stoppage of the process. Severity Level I 

lessons will require a response from the site to PMCSD.



Severity Level II - Could adversely affect reliability, operability or 

productivity or cause limited damage to equipment, facilities or 

temporary process shutdown. Severity Level II requires engineering/

managerial judgment to determine whether a response from the site will 

be required.



Severity Level III - Could result in minimal damage with minimal 

monetary cost to repair or replace. Severity level III lessons do not 

require a site response and are assigned to the site to determine the 

disposition of the lesson.



Each site develops its plan of action including a schedule to 

accomplish the plan; this is reported to the PMCSD. Each site validates 

the workability of the solution. At this time sites do not report back 

through the Programmatic Lessons Learned Program; however, the sites do 

monitor all Engineering Change Proposals and lessons learned. Under a 

new effort currently being developed the systems contractors will 

assume more responsibility for the prioritization, monitoring, and 

validation of lessons learned that are of programmatic interest. The 

System Contractors will report this information to applicable PMCD 

managers through a more efficient lessons learned information 

management system.



c. Improve the Organization Structure of the Database. Concur:



The organizational structure of the Programmatic Lessons Learned 

database has been improved, making it easier to locate information. The 

Programmatic Lessons Learned database is currently formatted into a 

Lessons Database and an Issues Database. The Lessons Database was 

developed in FY01-02 for the specific purpose of making it easier to 

locate information. Since the Lessons Database is relatively new 

(development was finished in February 2002) current users may not be 

familiar with it yet. All lessons are categorized at the highest level 

by life cycle phases of a chemical demilitarization plant (i.e. Design, 

Construction, Systemization, Operations, and Closure). Each phase is 

then broken down into subcategories creating a category tree by which 

users can quickly navigate to the area of interest.



A further enhancement is underway to convert the current Programmatic 

Lessons Learned database to an Intemet-based program. This project is 

scheduled for completion during the 2nd quarter, FY03.



RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 

direct the Secretary of the Army to develop policies and procedures for 

capturing and sharing lessons on an ongoing basis with the Alternative 

Technology and Approaches Project and in consultation with the Under 

Secretary of Defense (Acquisition, Technology and Logistics) develop 

policies and procedures for capturing and sharing lessons on an ongoing 

basis with the Assembled Chemical Weapons Assessment Program.



ARMY RESPONSE: Concur.



PMCSD and the Project Manager for Alternative Technologies and 

Approaches (PMATA) have made some progress toward sharing lessons 

learned. PMATA has participated in the PMCSD Engineering Change 

Proposal Review Process meetings and reviewed the Programmatic Lessons 

Learned database for applicable lessons learned. The two sites under 

the PMATA’s responsibility have benefited from the lessons learned 

database by avoiding previously identified programmatic shortcomings. 

As key milestones are met, PMATA’s increased participation in the 

Lessons Learned Program will be encouraged.



The Lessons Database was completed in August 2001. Since then, the 

information in the Lessons Database, originally compiled under the 

Programmatic Lessons Learned Program, has been shared with the Program 

Manager for the Assembled Chemical Weapons Assessment (PMACWA). 

Continued and increased use of the Database will be encouraged.



[End of section]



FOOTNOTES



[1] Chemical Weapons: FEMA and Army Must Be Proactive in Preparing 

States for Emergencies GAO-01-850 (Washington, D.C., Aug. 13, 2001).



[2] DOD, in its Knowledge Management Primer, provides managers and 

practitioners with a framework for sharing knowledge. The Army uses 

knowledge management principles in its Roadmap for Army Knowledge 

Management.



[3] In our first report, we recommended that the Chemical Stockpile 

Emergency Preparedness Program be more proactive, i.e., it should share 

its lessons learned--especially those concerning emergency readiness 

and response--with other stakeholders. This would include the Chemical 

Stockpile Disposal Project.



[4] Although Pine Bluff, Arkansas, is an incineration site, the Army is 

considering destroying a portion of the agent stored at Pine Bluff by 

using an alternative method under the Alternative Technologies and 

Approaches Project. No decision on whether an alternative technology 

will be used at the Pine Bluff site has been reached.



[5] The Senate ratified the U.N.-sponsored Convention on the 

Prohibition of the Development, Production, Stockpiling and the Use of 

Chemical Weapons and on Their Destruction (known as the Chemical 

Weapons Convention) in April 1997. 



[6] In accordance with provisions of the treaty, the Army states that 

an extension of the April 2007 deadline will be requested if and when 

necessary.



[7] A lessons learned process is considered an integral part of most 

knowledge management systems. 



[8] U.S. Department of Energy Standard: Corporate Lessons Learned 

Program Guidance (DOE-STD-7501-99).



[9] After the agent is destroyed in the incinerator, the pollution 

abatement system cleans the air (gases produced during incineration) 

before it is released into the environment. 



[10] Initially the Johnston Atoll site reported experiencing 

installation problems with the lower cost material. Later, however, 

both the Tooele and Johnston Atoll sites informed the Lessons Learned 

Program that a more expensive material (hastelloy) was the appropriate 

solution to address the piping failures. 



[11] Program Manager for Chemical Demilitarization headquarters 

management made the decision to continue using the lower cost material 

in part of the pollution abatement piping systems at the three future 

sites; the recommended lesson emerged from a repeated problem. 

Implementing the recommended lesson would have cost the Chem-Demil 

Program more than $750,000 and involve multiple sites.



[12] Military Training: Potential to Use Lessons Learned to Avoid Past 

Mistakes Is Largely Untapped , D.C., Aug. 9, 1995).



[13] U.S. Department of Energy Standard: Corporate Lessons Learned 

Program Guidance (DOE-STD-7501-99). According to the standard, the 

development process includes identification, documentation, 

validation, and dissemination. The utilization and incorporation 

process includes identification of applicable lessons, distribution to 

appropriate personnel, identification of actions that will be taken as 

a result of the lessons, and follow-up to ensure that appropriate 

actions were taken. 



[14] See footnote 8.



[15] In January 2002, we reported on problems related to the knowledge 

management database tool used by the National Aeronautics and Space 

Administration’s lessons learned program, see NASA: Better Mechanisms 

Needed for Sharing Lessons Learned, GAO-02-195 (Washington, D.C., Jan. 

30, 2002).



[16] The Non-Stockpile Chemical Materiel Product maintains a separate 

lessons learned database that is linked to the Lessons Learned 

Program’s database. The Chemical Stockpile Emergency Preparedness 

Program maintains its own best practices on an Internet site, shares 

lessons at national meetings, and does coordinate with the Lessons 

Learned Program, especially for outreach and public relations efforts. 



[17] Chemical Weapons Disposal: Improvements Needed in Program 

Accountability and Financial Management (GAO/NSIAD-00-80, May 8, 2000).



[18] The Assembled Chemical Weapons Assessment Program submitted a 

formal request for lessons and cost data through the Deputy Assistant 

to the Secretary of Defense (Chemical/Biological Defense) and the 

Deputy Assistant Secretary of the Army (Chemical Demilitarization).



[19] In this report, we refer to this population as “surveyed staff” to 

distinguish from the general interviews. 



[20] 10 U.S.C. 2430.



[21] Omnibus Consolidation Appropriations Act, 1997 (P.L. 104-208).



[22] The Under Secretary of Defense (Acquisition and Technology) is now 

titled the Under Secretary of Defense (Acquisition, Technology, and 

Logistics).



[23] The Cooperative Threat Reduction program, which assists Russia in 

destroying over 40,000 tons of chemical agent stored there, is part of 

the Chemical Demilitarization Program’s mission but is funded 

separately.



[24] In August 2001, we reported that the Chemical Stockpile Emergency 

Preparedness Program did not share its lessons effectively. During this 

review, the program’s management demonstrated steps that had been taken 

to address this issue. 



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: