This is the accessible text file for GAO report number GAO-10-449 
entitled 'Agency for Toxic Substances and Disease Registry: Policies 
and Procedures for Public Health Product Preparation Should Be 
Strengthened' which was released on May 20, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 
GAO: 

April 2010: 

Agency for Toxic Substances and Disease Registry: 

Policies and Procedures for Public Health Product Preparation Should 
Be Strengthened: 

GAO-10-449: 

GAO Highlights: 

Highlights of GAO-10-449, a report to congressional requesters. 

Why GAO Did This Study: 

The Agency for Toxic Substances and Disease Registry (ATSDR) has faced 
concerns related to the quality of some of the public health products 
it publishes. ATSDR investigates community exposures related to 
certain hazardous chemical sites and releases; assesses associated 
health effects; and recommends actions to stop, prevent, or minimize 
harmful effects. ATSDR publishes many types of products, including 
public health assessments, health consultations, exposure 
investigations, and health study reports. GAO was asked to examine the 
extent to which ATSDR’s policies and procedures for product 
preparation, including work initiation, product development, and 
review and clearance, provide reasonable assurance of product quality. 
GAO reviewed ATSDR policies and procedures and interviewed agency 
officials and employees. 

What GAO Found: 

The policies and procedures that ATSDR has established for public 
health product preparation lack some of the critical controls to 
provide reasonable assurance of product quality. To provide reasonable 
assurance that agency objectives are being met, federal internal 
control standards call for agencies to establish policies and 
procedures, assess risks associated with achieving agency objectives, 
ensure effective information sharing throughout the organization, 
monitor agency activities, and establish key areas of authority and 
responsibility for management and staff. While ATSDR has established 
some policies and procedures to guide the preparation of its public 
health products, the policies and procedures do not establish how 
information is to flow between management and staff during initiation. 
Absent such policies and procedures, ATSDR generally relies on various 
meetings to inform management and staff about new work. The agency is 
also implementing a new database, which may improve information flow. 
Furthermore, ATSDR does not comprehensively evaluate and categorize 
the risk of work being initiated. While the agency used to officially 
classify some hazardous chemical sites as “high-priority” or “focus 
sites,” and require any products resulting from those sites to undergo 
a higher level of review and clearance, it no longer does so. Because 
ATSDR does not comprehensively assess and categorize the risk of work 
being initiated at the agency, management cannot ensure that they have 
consistently managed the risk related to new work. 

Additionally, many of ATSDR’s policies and procedures that guide 
product development do not clearly define management roles and 
responsibilities and do not consistently require that management 
monitor the development of key components of these products. These 
deficiencies may lead management to be unclear about their 
responsibilities, and may result in problems that occur during product 
development not being identified or addressed until review and 
clearance, if at all. For example, ATSDR and Institute of Medicine 
reports show that because scientific concerns were not identified 
during development of an ATSDR report regarding chemical releases in 
the Great Lakes region, the document underwent several years of 
review, and a final report was not issued until more than 4 years 
after the first draft was written. 

Moreover, because some review and clearance policies do not reflect 
current practices, ATSDR staff cannot rely on these policies to 
accurately or consistently determine review and clearance procedures. 
Furthermore, review and clearance policies and procedures direct 
management and staff to use discretion to identify products that 
require higher levels of review, rather than making this determination 
through a comprehensive risk assessment process. While ATSDR policy 
sets out criteria for when additional review may occur, such as when a 
document could have a high degree of visibility, there is no required 
point during a product’s preparation when management and staff 
collectively determine whether a product meets the criteria, and 
whether additional review is warranted. Thus, the agency cannot ensure 
that all products consistently receive the appropriate level of review. 

What GAO Recommends: 

GAO recommends that ATSDR develop policies and procedures that direct 
management to assess the risk level of work when it is initiated and 
reevaluate the risk level throughout product preparation to ensure it 
remains appropriate, and that ATSDR revise its policies and procedures 
to include guidance about management’s roles and responsibilities in 
monitoring product development. ATSDR stated that it has begun to 
incorporate GAO’s recommendations. 

View GAO-10-449 or key components. For more information, contact 
Cynthia A. Bascetta at (202) 512-7114 or bascettac@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

ATSDR's Policies and Procedures for Public Health Product Preparation 
Lack Some Critical Controls to Provide Reasonable Assurance of Quality: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Comments from the Agency for Toxic Substances and Disease 
Registry: 

Appendix II: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: ATSDR Organizational Structure: 

Figure 2: ATSDR Regions and Cooperative Agreement Partners: 

Abbreviations: 

ATSDR: Agency for Toxic Substances and Disease Registry: 

CDC: Centers for Disease Control and Prevention: 

CERCLA: Comprehensive Environmental Response, Compensation, and 
Liability Act of 1980: 

DHAC: Division of Health Assessment and Consultation: 

DHS: Division of Health Studies: 

DRO: Division of Regional Operations: 

DTEM: Division of Toxicology and Environmental Medicine: 

EPA: Environmental Protection Agency: 

FEMA: Federal Emergency Management Agency: 

HHS: Department of Health and Human Services: 

NCEH: National Center for Environmental Health: 

NIEHS: National Institute of Environmental Health Sciences: 

NIH: National Institutes of Health: 

NPL: National Priorities List: 

OD: Office of the Director: 

PHAGM: Public Health Assessment Guidance Manual: 

SARA: Superfund Amendments and Reauthorization Act of 1986: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

April 30, 2010: 

The Honorable Bart Gordon: 
Chairman: 
Committee on Science and Technology: 
House of Representatives: 

The Honorable Brad Miller: 
Chairman: 
The Honorable Paul C. Broun: 
Ranking Member: 
Subcommittee on Investigations and Oversight: 
Committee on Science and Technology: 
House of Representatives: 

The Honorable F. James Sensenbrenner: 
House of Representatives: 

The Agency for Toxic Substances and Disease Registry (ATSDR)[Footnote 
1] has faced concerns related to the quality of some of the public 
health products it publishes. ATSDR is responsible for investigating 
community exposures related to certain hazardous chemical sites and 
releases; assessing associated health effects; recommending actions to 
stop, prevent, or minimize harmful effects; and conducting health 
studies.[Footnote 2] In conducting these activities, the agency 
publishes many types of public health products, including public 
health assessments, health consultations, health study reports, and 
exposure investigations. In 1991, an expert panel we convened 
conducted a detailed evaluation of the quality of a sample of 15 ATSDR 
public health assessments and found that there were deficiencies with 
these products.[Footnote 3] More recently, reports by the Institute of 
Medicine[Footnote 4] and ATSDR's Board of Scientific Counselors 
[Footnote 5] have identified various concerns such as the 
appropriateness and quality of the data used in ATSDR's products, the 
methodology and design of the studies, clearance policies, and the use 
of external peer review and response to review comments. 

Some members of the Congress have also expressed concern about the 
quality of ATSDR's public health products. On April 1, 2008, the U.S. 
House of Representatives' Subcommittee on Investigations and 
Oversight, Committee on Science and Technology, held a hearing to 
examine ATSDR's handling of the preparation of a health consultation 
on the formaldehyde levels in trailers that the Federal Emergency 
Management Agency (FEMA) provided to victims of Hurricanes Katrina and 
Rita. The health consultation, which was published in February 2007, 
raised congressional concerns about the quality of ATSDR's products 
and the involvement of agency leadership in the issuance of a flawed 
product.[Footnote 6] In response to the hearing and further 
examination of ATSDR's role in the FEMA trailers health consultation, 
the subcommittee issued a report in September 2008 to express its 
heightened concern that ATSDR was issuing public health products of 
poor quality.[Footnote 7] On March 12, 2009, the subcommittee held a 
second hearing--The Agency for Toxic Substances and Disease Registry 
(ATSDR): Problems in the Past, Potential for the Future?--which 
focused on its continued concern about the quality of ATSDR's products. 

ATSDR has responded to these concerns, noting that multiple factors 
have posed challenges for the agency, including limitations in the 
ability of available science to answer community questions about the 
effect of chemical exposures, limitations in ATSDR's ability to 
collect data related to exposures, and reductions since 2004 in the 
number of ATSDR staff and resources available to conduct the agency's 
mission. In his testimony for the March 2009 congressional hearing, 
ATSDR's director at that time[Footnote 8] noted that the agency 
intended to reexamine its approach to carrying out its mission in 
light of these challenges, and had convened a National Conversation on 
Public Health and Chemical Exposures which includes government, 
community groups, and industry to create an agenda for revitalizing 
the public health approach to chemical exposures, which would include 
future direction for ATSDR.[Footnote 9] 

You have expressed interest in ensuring the quality of ATSDR's public 
health products. In this report we examine the extent to which ATSDR's 
policies and procedures for product initiation, development, and 
review and clearance provide reasonable assurance of public health 
product quality. 

To do our work, we reviewed ATSDR's policies and procedures and 
interviewed officials to identify guidance related to the preparation 
of public health products. Preparation of public health products 
encompasses (1) initiation, which includes a decision by the agency to 
begin work on a public health product and the assignment of staff to 
prepare the product, (2) development, which includes management 
approval to proceed with the development of a product and the actual 
drafting of the public health product, and (3) review and clearance, 
which is the process by which a product is internally or externally 
reviewed and disseminated as a final public health product.[Footnote 
10] We focused our review of ATSDR's policies and procedures on those 
related to public health assessments, health consultations, exposure 
investigations, and health study reports[Footnote 11] because these 
products are considered core public health products by ATSDR and 
concerns have been raised about the quality of products such as these, 
in which ATSDR identifies potential exposures to hazardous chemicals 
and assesses associated health effects. Throughout this report, we use 
the phrase "public health products" to refer solely to those products 
on which we focused our review: public health assessments, health 
consultations, exposure investigations, and health studies. We 
compared the policies and procedures ATSDR uses to guide the 
preparation of its public health products to the standards described 
in the Standards for Internal Control in the Federal Government 
[Footnote 12],[Footnote 13] and the related Internal Control 
Management and Evaluation Tool.[Footnote 14] We did not evaluate 
ATSDR's policies and procedures on human capital, financial 
management, or scientific and technical risk assessment. Additionally, 
we did not review ATSDR products to assess their quality. Accordingly, 
we do not express any view about their accuracy, completeness, or 
scientific credibility. 

In addition, to gain a better understanding of ATSDR and the policies 
and procedures related to product preparation, we conducted a series 
of small group interviews with ATSDR team leads[Footnote 15] and 
nonmanagement employees in ATSDR's Headquarters with responsibilities 
involving the preparation of public health products. Each small group 
interview consisted of a group discussion to capture general themes 
and opinions related to the policies and procedures. We conducted a 
total of six small group interviews in ATSDR Headquarters, with no 
more than eight employees in each interview. These small group 
interviews included one meeting with team leads from various ATSDR 
divisions; one meeting each with employees from the Division of 
Toxicology and Environmental Medicine (DTEM), which participates in 
preparing health consultations and provides technical expertise during 
emergencies, and employees from the Division of Health Studies (DHS), 
which is primarily responsible for the preparation of health study 
reports; and one meeting with technical project officers who oversee 
the work of cooperative agreement partners.[Footnote 16] Additionally, 
we held two small group interviews with employees from the Division of 
Health Assessment and Consultation (DHAC), which is primarily 
responsible for the preparation of public health assessments, health 
consultations, and exposure investigations. For the small group 
interview with team leads, we selected employees who were identified 
as team leads on the October 2009 personnel roster as well as 
employees ATSDR management identified who performed the duties of team 
leads. For all other small group interviews with nonmanagement 
employees, we randomly selected individuals to interview from a 
population of nonmanagement employees in each division obtained from 
the October 2009 personnel roster.[Footnote 17] We submitted all the 
names of the randomly selected individuals to ATSDR to ensure that 
these individuals did not perform management duties and to coordinate 
their availability for the interviews. Some employees who were 
selected to participate in the small group interviews were not able to 
attend. In total, we interviewed 33 team leads and nonmanagement 
employees. To encourage the candor of the individuals who participated 
in the small group interviews, we did not record their names in our 
notes from those interviews and agreed not to share our notes with 
ATSDR management. Additionally, at the conclusion of each of the six 
small group interviews we administered a short questionnaire to the 
participants to collect additional information about their 
perspectives on the policies and procedures that guide their work, and 
on improving public health product quality at the agency. Of the 33 
questionnaires we distributed, we received 30 completed 
questionnaires. The views expressed by these employees cannot be 
generalized to all employees working within these divisions or in 
these roles. 

We also conducted six on-site small group interviews. We interviewed 
employees in 3 of ATSDR's 10 Division of Regional Operations (DRO) 
offices, and employees in 3 of 30 cooperative agreement partner 
offices.[Footnote 18] We chose the three regions that issued the 
greatest number of public health assessments and health consultations 
in fiscal year 2008. Those regions were Region 5 (Illinois, Indiana, 
Michigan, Minnesota, Ohio, and Wisconsin), Region 10 (Alaska, Idaho, 
Oregon, and Washington), and Region 4 (Alabama, Florida, Georgia, 
Kentucky, Mississippi, North Carolina, South Carolina, and Tennessee). 
We interviewed the three cooperative agreement partners that were 
located in the same states as our selected regional offices--the 
Washington State Department of Health, the Illinois Department of 
Public Health, and the Georgia Division of Public Health. 

We also conducted interviews with officials, experts, and researchers 
outside ATSDR to gain an understanding of ATSDR's relationship with 
other agencies, to get their perspectives on ATSDR's work, and to 
learn about the policies and procedures used by other prominent 
scientific research organizations. We conducted interviews with 
federal officials from the Centers for Disease Control and Prevention 
(CDC); the Environmental Protection Agency (EPA), which ATSDR advises 
about the health aspects of hazardous waste sites or spills; and the 
National Institutes of Health's (NIH) National Institute of 
Environmental Health Sciences (NIEHS), with which ATSDR collaborates 
on various matters related to environmental health science. We also 
conducted interviews with officials from two national scientific 
research organizations, the National Academies and the National 
Science Foundation. We interviewed two experts in environmental health 
science who had experience working with ATSDR, and we spoke with two 
advocacy organizations that work with communities that have been 
affected by environmental health problems. 

We conducted this performance audit from April 2009 to April 2010, in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

ATSDR investigates community exposures related to chemical sites and 
releases; works with federal, tribal, state, and local agencies to 
identify potential exposures; assesses associated health effects; and 
recommends actions to stop, prevent, or minimize these harmful 
effects, among other things. 

ATSDR History: 

ATSDR was established by the Comprehensive Environmental Response, 
Compensation, and Liability Act of 1980 (CERCLA), which created what 
is known as the Superfund program to clean up the nation's most 
dangerous hazardous waste sites.[Footnote 19] CERCLA established ATSDR 
to carry out Superfund's health-related activities, including the 
establishment of a national registry of and the provision of medical 
care and testing to persons exposed to toxic substances and the 
provision of survey and screening programs to determine the 
relationship between such exposure and illness.[Footnote 20] In 1985, 
ATSDR was formally organized to begin to carry out its 
responsibilities under the law. The Superfund Amendments and 
Reauthorization Act of 1986 (SARA) broadened ATSDR's responsibilities 
to include, among other things, public health assessments, 
establishment and maintenance of toxicologic databases, information 
dissemination, and health education. SARA required that ATSDR conduct 
a public health assessment at each site proposed for or on the 
National Priorities List (NPL),[Footnote 21] and authorized ATSDR to 
perform public health assessments upon petition by an individual or 
physician and to conduct additional follow-up health studies if 
needed.[Footnote 22] 

Sources of ATSDR Work and Types of Public Health Products: 

ATSDR may initiate work to prepare public health products for several 
reasons. Work can be necessitated pursuant to SARA by a site's 
proposal to or listing on the NPL, requested by an ATSDR partner such 
as EPA, negotiated as part of a work plan for federal facilities, 
petitioned by individuals or physicians, or generated internally by 
ATSDR officials. Once work is initiated, ATSDR may prepare any of 
several different types of products, including the following: 

* Public health assessments evaluate data and information on the 
release of hazardous substances into the environment in order to 
assess any past, current, or future impact on public health, develop 
health advisories or other recommendations, and identify studies or 
actions needed to evaluate and mitigate or prevent human health 
effects. 

* Health consultations review available information or collect new 
data to respond to a specific health question or request for 
information about a potential environmental hazard. Health 
consultations are focused on a specific exposure issue and provide 
guidance on the specific health-related question. 

* Health studies are epidemiological research conducted to investigate 
and characterize the association between exposure to chemicals in the 
environment and health problems of people who have been exposed to 
chemicals. 

* Exposure investigations collect and analyze site-specific 
environmental or biological samples to determine whether individuals 
have been exposed to hazardous substances. Exposure investigations are 
often designed to examine individuals most likely to be exposed to 
hazardous substances, rather than a sample of individuals from the 
exposed community that would provide information about the community 
as a whole.[Footnote 23] 

The time required to complete ATSDR public health products varies, and 
may depend on the nature and complexity of the work site. For example, 
some public health assessments and most health studies take one or 
more years to complete, whereas some health consultations are 
completed within weeks. ATSDR also prepares emergency response 
products--most of which are completed within hours or days--which are 
intended to help interpret the implications of exposure data. These 
"real-time" investigations of health exposures include technical 
assistance and health consultations, and are often initiated in 
response to requests from agencies such as EPA or state health or 
environmental departments. 

ATSDR Organizational Structure: 

Although ATSDR was established within the Department of Health and 
Human Service's (HHS) Public Health Service, the Director of CDC 
serves as the Administrator of ATSDR, and CDC performs many 
administrative functions for ATSDR, such as human capital and 
financial management services. ATSDR is located within CDC's Office of 
Noncommunicable Diseases, Injury and Environmental Health. In 2003, 
ATSDR's administrative functions were combined with those of CDC's 
National Center for Environmental Health (NCEH). ATSDR and NCEH share 
an Office of the Director (OD), which is led by a director and deputy 
director.[Footnote 24] ATSDR has four divisions, each of which is 
divided into either branches, programs, or regional offices (see fig. 
1). Each division is led by a director and deputy director, and DHAC, 
DHS, and DTEM each also have an associate director for science. Each 
branch or program within a division is led by a chief. 

Figure 1: ATSDR Organizational Structure: 

[Refer to PDF for image: Organizational chart] 

Top level: CDC Director: 

Second level, reporting to CDC Director: 
Office of Noncommunicable Diseases, Injury and Environmental Health: 

Third level, reporting to Office of Noncommunicable Diseases, Injury 
and Environmental Health: 
ATSDR: Office of the Director (OD)[A]: 
* Communication Science; 
* Community Environmental Health; 
* Financial and Administrative Services; 
* Information Systems; 
* Policy, Planning and Evaluation; 
* Program Development; 
* Terrorism Preparedness and Emergency Response; 
* Science. 

Fourth level, reporting to ATSDR: Office of the Director (OD): 

Divisions: 

Division of Health Assessment and Consultation (DHAC); 
Branch/Regional Offices: 
* Cooperative Agreement and Program Evaluation Branch; 
* Exposure Investigations and Site Assessment Branch; 
* Health Promotion and Community Involvement Branch; 
* Site & Radiological Assessment Branch. 

Division of Health Studies (DHS); 
Branch/Regional Offices: 
* Geographic Research, Analysis and Services Program; 
* Health Investigations Branch; 
* Surveillance and Registries Branch. 

Division of Regional Operations (DRO); 
Branch/Regional Offices: 
* ATSDR Liaison Office EPA Headquarters/Washington, DC; 
* 10 Regional Offices[B]. 

Division of Toxicology and Environmental Medicine (DTEM); 
Branch/Regional Offices: 
* Applied Toxicology Branch; 
* Computational Toxicology Methods and Development Laboratory; 
* Environmental Medicine and Educational Services Branch; 
* Prevention, Response and Medical Support Branch. 

Source: GAO analysis of CDC and ATSDR information. 

[A] While the National Center for Environmental Health (NCEH) and 
ATSDR share the same OD, the organizations perform different 
functions. Our review focused only on those public health products 
produced by ATSDR. Therefore, we have not included NCEH or its 
divisions in this figure. 

[B] The 10 regional offices are located in Boston, New York, 
Philadelphia, Atlanta, Chicago, Dallas, Kansas City, Denver, San 
Francisco, and Seattle. 

[End of figure] 

OD: As shown in figure 1, the OD has eight functional areas: 
communication science; community environmental health; financial and 
administrative services; information systems; policy, planning and 
evaluation; program development; science; and terrorism preparedness 
and emergency response. These functional areas are responsible for 
providing scientific and programmatic support for agency staff and 
conducting review and clearance for public health products produced by 
ATSDR divisions. Specifically, the Office of Science is responsible 
for the clearance, cross-clearance,[Footnote 25] and external peer 
review of ATSDR public health products. The Office of Science also 
coordinates the NCEH/ATSDR Board of Scientific Counselors, which 
provides advice and guidance to ATSDR's director on external peer 
review of ATSDR programs and issues including program goals, 
objectives, strategies, and priorities.[Footnote 26] The board's 
advice and guidance are intended to assist ATSDR in ensuring 
scientific quality, timeliness, utility, and dissemination of 
scientific results. 

DHAC: DHAC produces a number of products, including public health 
assessments, health consultations, and exposure investigations. DHAC's 
Cooperative Agreement and Program Evaluation Branch is charged with 
supporting and overseeing the work produced by ATSDR's cooperative 
agreement partners, which currently include 29 state agencies and 1 
tribal government (see fig. 2). In order to become a cooperative 
agreement partner, state and tribal governments must respond to a 
request for applications that ATSDR posts, and have their application 
reviewed, scored, and funded by ATSDR. Currently, ATSDR funds 
cooperative agreement partners for a 5-year funding cycle.[Footnote 
27] Through these partnerships, ATSDR provides funding and technical 
support for state and tribal government employees to assess 
environmental health concerns at sites within their jurisdictions and 
to conduct needed public health interventions. Cooperative agreement 
partners prepare public health products that are monitored, reviewed, 
and cleared by ATSDR. DHAC staff work with staff in regional offices 
to provide technical assistance to cooperative agreement partners and 
the public, and to sponsor activities in communities that have been 
exposed to hazardous chemicals. 

Figure 2: ATSDR Regions and Cooperative Agreement Partners: 

[Refer to PDF for image: 2 U.S. maps] 

ATSDR regions: map indicates geographic borders of the 10 ATSDR 
regions. 

States in each ATSDR region with cooperative agreements: 

Region 1: 
Connecticut; 
Massachusetts; 
New Hampshire. 

Region 2: 
New Jersey; 
New York. 

Region 3: 
Pennsylvania; 
West Virginia. 

Region 4: 
Florida; 
Georgia; 
North Carolina; 
Tennessee. 

Region 5: 
Illinois; 
Michigan; 
Minnesota; 
Ohio; 
Wisconsin. 

Region 6: 
Arkansas; 
Louisiana; 
Texas. 

Region 7: 
Iowa; 
Missouri. 

Region 8: 
Colorado; 
Utah. 

Region 9: 
Arizona (Gila River Tribe); 
California. 

Region 10: 
Alaska; 
Idaho; 
Oregon; 
Washington. 

Source: GAO analysis of CDC and ATSDR information. 

[End of figure] 

DHS: DHS is responsible for conducting epidemiologic health studies, 
designing and conducting surveillance programs, and establishing and 
maintaining registries. The division collects information to determine 
whether a chemical exposure is making people sick, and collects data 
on persons identified as having been exposed to a specific contaminant 
or event. DHS is also involved with the ongoing collection, analysis, 
and interpretation of health data. These products are released as 
ATSDR reports and are often published in the scientific literature. 

DRO: DRO staffs an ATSDR regional office within each of the 10 EPA 
regions, in EPA Headquarters in Washington, D.C., and in two satellite 
offices in Helena, Montana, and Anchorage, Alaska (see fig. 2). These 
offices are responsible for acting as regional liaisons to establish 
working relationships with EPA, other federal and state agencies, 
individual citizens, and community groups to maintain current and 
historic knowledge of issues related to hazardous chemical sites in 
their regions. Working in collaboration with DHAC, DRO staff also 
prepare a number of time-sensitive products, such as health 
consultations. 

DTEM: DTEM assists in the production of health consultations and 
provides technical assistance in response to chemical spills and acute 
events. DTEM staff also work closely with DRO staff to provide real- 
time public health advice in case of a chemical release. DTEM is 
responsible for serving as a resource for information and assistance 
on toxic substances in the environment, and prepares toxicological 
profiles[Footnote 28] for hazardous substances. 

Internal Control: 

The Standards for Internal Control in the Federal Government provides 
the overall framework for establishing guidelines for internal 
control, and helps government managers achieve desired objectives. 
[Footnote 29] As applied to ATSDR, this could include the preparation 
of quality public health products. Internal control, which is 
synonymous with management control, comprises the plans, methods, and 
procedures used to meet missions, goals, and objectives. The related 
Internal Control Management and Evaluation Tool assists agencies in 
maintaining or implementing effective control.[Footnote 30] Internal 
control is not one event, but a series of actions and activities that 
occur throughout an entity's operations and on an ongoing basis. The 
responsibility of good internal control rests with all managers; they 
set the objectives, put the control mechanisms and activities in 
place, and monitor and evaluate these mechanisms and activities. 
However, all employees in the organization play important roles in 
this process. Internal control includes activities such as 
establishing policies and procedures, assessing risks associated with 
achieving agency objectives, ensuring effective information sharing 
throughout the organization, conducting ongoing monitoring of agency 
activities, and establishing key areas of authority and responsibility 
for agency management and staff. 

The Standards note, however, that while internal control helps 
government managers achieve desired objectives, it cannot provide 
absolute assurance that all agency objectives will be met. There are 
many factors outside the control and influence of management that can 
affect an agency's ability to achieve its objectives. For example, 
human mistakes, judgment errors, and acts of collusion to circumvent 
control can affect meeting agency objectives. Therefore, once in 
place, internal control provides reasonable assurance, not absolute 
assurance, that an agency's objectives are being achieved. 

ATSDR's Policies and Procedures for Public Health Product Preparation 
Lack Some Critical Controls to Provide Reasonable Assurance of Quality: 

The policies and procedures that ATSDR has established for public 
health product preparation lack some of the critical controls to 
provide reasonable assurance of product quality. The controls that 
have not been incorporated involve information flow, risk assessment, 
and management roles, responsibilities, and monitoring. Although the 
agency has established some policies and procedures to govern 
initiation of work to prepare public health products, ATSDR lacks 
policies and procedures that (1) establish how information about newly 
initiated work should effectively flow between all levels of 
management and staff, and (2) describe how to comprehensively assess 
and categorize the risk of work being initiated at the agency. 
Additionally, while some policies and procedures state the roles of 
staff in product development, many do not identify the roles and 
responsibilities of management. Moreover, although policies and 
procedures include some routine oversight of product development, they 
do not consistently require that management monitor the development of 
key components of these products. Finally, while ATSDR has implemented 
policies and procedures governing review and clearance, some sections 
of the policies and procedures do not reflect current practices, and 
the policies and procedures do not ensure that all products 
consistently receive appropriate review. 

ATSDR's Policies and Procedures for Work Initiation Do Not Establish 
and Describe Information Flow or Adequate Assessment of Risk: 

Although ATSDR has established some policies and procedures to govern 
the initiation of work to prepare public health products, its policies 
and procedures do not establish and describe how information about 
newly initiated work should flow between all levels of management and 
staff. Consequently, the agency cannot be certain that all management 
and staff have the information they need to do their jobs effectively. 
The Standards for Internal Control in the Federal Government states 
that for an entity to run and control its operations, it must have 
relevant, reliable, and timely communications relating to internal as 
well as external events. Information is needed throughout the agency 
to achieve all of its objectives, and effective communications should 
occur in a broad sense with information flowing down, across, and up 
the organization. 

Since ATSDR has not established policies or procedures on how 
information about newly initiated work should flow between all levels 
of management and staff, it generally relies on various meetings held 
at different levels throughout the agency to inform management and 
staff about newly initiated work. ATSDR officials stated that when 
site activities are controversial or of special interest, management 
is informed through weekly Issues Management meetings and Senior Staff 
meetings between ATSDR's OD and division directors. The former ATSDR 
director said that product initiation had an important role in the 
Issues Management meetings. According to ATSDR officials, information 
from these meetings is shared with mid-and lower-level management 
through notes and face-to-face meetings. Officials stated that newly 
initiated work may also be discussed during other regular meetings 
within the divisions and branches. 

In addition to a lack of policies or procedures on information flow, 
for several years ATSDR has operated with fragmented databases in 
which information about newly initiated work is entered and tracked, 
none of which are accessible to, or ensure information flows to, 
people at all levels of the agency. ATSDR previously used a tracking 
system called HazDat, which was taken off line in 2007.[Footnote 31] 
Since that time, ATSDR management and staff have been without an 
agencywide system that is capable of providing information about newly 
initiated work to people at all levels of the agency. There are 
several other agency databases that contain information about newly 
initiated work, but none of these systems are accessible to people at 
all levels of the agency. Examples of these databases include a DHAC 
Tracking and Triage Database and a Petition Database.[Footnote 32] 
However, ATSDR officials told us that access to the DHAC Tracking and 
Triage Database was limited to management and staff within DHAC, and 
that the Petition Database was accessible by only three agency 
employees.[Footnote 33] Additionally, ATSDR officials told us that no 
regularly scheduled reports were generated from these databases, 
although division management was provided with a weekly update on 
petitions under review.[Footnote 34] ATSDR's OD also created a 
database about 2 years ago to track issues discussed during the weekly 
Issues Management meetings. However, while the Issues Management 
database is used as a tool during the meetings, it is not accessible 
to division directors or other ATSDR management and staff outside of 
the meeting. 

Although ATSDR has not established policies or procedures that 
establish and describe information flow within the agency, it is 
implementing a new agencywide system called Sequoia, which may improve 
the flow of information about newly initiated work between management 
and staff.[Footnote 35] While data entry into Sequoia began in 2007, 
the former director of ATSDR told us that resource limitations slowed 
Sequoia's development and that the use of fragmented databases was a 
temporary measure until Sequoia was completed.[Footnote 36] However, 
officials told us that while they expected that Sequoia would replace 
other existing databases, further evaluation is needed to determine if 
Sequoia could do everything required by management or if some 
information will still have to be captured in separate systems. ATSDR 
officials told us that Sequoia was designed to track requests, cost 
recovery reimbursement, exposure data, work flow for site-specific 
products, and information pertaining to other products or services 
done on particular sites. Sequoia includes some major features that 
were not available in previous ATSDR systems, such as providing a 
centralized database that is available to all ATSDR staff for tracking 
incoming work requests, and providing a system for reporting and 
retrieving information on the public health impact and outcome of 
public health activities. According to agency officials, in January 
2010 ATSDR employees began using Sequoia for planning site and project 
activities, recording the results of their investigative and community 
outreach efforts, and reporting the public health accomplishments of 
their activities. 

ATSDR also lacks comprehensive policies and procedures for assessing 
and categorizing the risk of work being initiated at the agency. The 
Standards for Internal Control in the Federal Government states that 
effective internal control should provide for an assessment of the 
risks the agency faces from both external and internal sources and 
that management needs to comprehensively identify risks and consider 
all significant interactions between the agency and other parties. 
Risk identification methods may include qualitative and quantitative 
ranking activities, management conferences, forecasting and strategic 
planning, and consideration of findings from audits and other 
assessments. Risk assessment also includes deciding how to manage the 
risk and what actions should be taken, and the Internal Control 
Management and Evaluation Tool notes that management should formulate 
an approach for risk management and decide on the internal control 
activities required to mitigate those risks.[Footnote 37] 

ATSDR previously incorporated some of the principles of risk 
assessment when the agency officially classified sites as "high 
priority" or "focus sites." ATSDR officials told us that these sites 
were typically identified by senior management and staff as those 
sites where chemical exposures may be of significant concern, which 
may require extensive agency resources, or may involve other site 
complexities. If a site was classified as a focus site, which 
typically occurred as work was being initiated, any public health 
products resulting from that site were required to undergo a higher 
level of review during review and clearance. However, ATSDR officials 
told us that they stopped using these classifications several years 
ago. Instead, agency officials and employees now use terms such as 
"high profile concern" or "sites of interest" to refer to those sites 
that might require additional review and clearance because they have 
high interest from the media or the Congress, or involve issues of 
difficult or emerging science. Officials stated that these sites were 
now managed through meetings such as the Issues Management meeting. 
They stated that they believed that the Issues Management process 
incorporated many of the principles of risk assessment by enabling 
senior agency management to identify and discuss important sites each 
week. Nevertheless, terms such as high profile are not official agency 
designations and do not trigger any additional required management 
monitoring during product development or required higher levels of 
review and clearance. Additionally, while certain high profile sites 
may be identified as they are initiated and discussed during Issues 
Management meetings, not all new sites are being reviewed by OD and 
division management to assess and categorize the risk to the agency of 
the public health products resulting from the sites. 

One ATSDR division, however, uses a process with elements similar to 
risk assessment in the way that it prioritizes work requests. DHAC 
generally uses a triage process for all ATSDR work requests requiring 
DHAC staff assistance. This process categorizes work requests as high, 
medium, or low priority. A request's priority level is based on three 
criteria, which in order of importance are extent of exposure, public 
health impact, and community and political interest, according to an 
ATSDR document explaining the DHAC triage process. A triage decision 
team, consisting of management-level staff from DHAC and DRO, decides 
on the priority level for the work request, and that information, 
along with other information about the request, is tracked in the DHAC 
Tracking and Triage Database. However, this information is used only 
to prioritize DHAC work requests and assign staff accordingly. This 
process is not used by other ATSDR divisions, and is not an official 
agency designation that triggers any additional requirements for that 
site or related public health products, such as additional management 
monitoring during product development or required higher levels of 
review and clearance. Because ATSDR does not currently have policies 
and procedures that describe how the agency is to comprehensively 
assess and categorize the risk of work it initiates to prepare public 
health products, management cannot ensure that it has consistently 
managed the risk related to all new work, or established product 
preparation procedures commensurate with the risk. 

ATSDR's Policies and Procedures for Product Development Do Not Provide 
for Clear Management Roles and Responsibilities or Consistent 
Monitoring of Product Development: 

While some of ATSDR's policies and procedures state the roles of staff 
in developing public health products, many do not identify the roles 
and responsibilities of management for ensuring that staff follow 
those policies and procedures. The Standards for Internal Control in 
the Federal Government states that management is responsible for 
developing the detailed policies, procedures, and practices to fit 
their agency's operations. The Standards states that the agency's 
organizational structure should clearly define key areas of authority 
and responsibility and establish appropriate lines of reporting. 
Internal control activities include approvals and the maintenance of 
related records to help ensure that management's directives are 
carried out. The Internal Control Management and Evaluation Tool also 
states that managers and supervisors need to know their 
responsibilities for internal control and need to make control and 
control monitoring part of their regular operating processes. 

The ATSDR documents that provide guidance on developing products do 
not clearly delineate management roles and responsibilities. The 
Public Health Assessment Guidance Manual (PHAGM) is the document that 
officials and employees of DHAC, DRO, and cooperative agreement 
partners identified as the primary document that guides their work. 
The PHAGM describes how to analyze site-specific data, make 
recommendations, and develop conclusion categories.[Footnote 38] This 
document is used by DHAC, DRO, and cooperative agreement staff to 
develop public health assessments, health consultations, and exposure 
investigations. The PHAGM guides staff in developing these products, 
but it does not establish lines of reporting or detail the 
responsibilities of management for monitoring product development. 
Additionally, although the PHAGM states that ATSDR promotes a team 
approach in conducting the public health assessment process, it does 
not describe how ATSDR management fits into this team approach. And 
while ATSDR officials stated that the PHAGM was not developed as a 
management guide, ATSDR does not have any other documents that provide 
guidance to management on their responsibilities for monitoring the 
development of public health assessments and health consultations. In 
addition to the PHAGM, there are a number of chemical-and exposure- 
specific and technical guidance documents that are used as 
supplements, as well as guidance specific to site work. These 
documents give additional information to staff on specific chemicals, 
how and when to use certain scientific methods, and site team 
procedures. However, like the PHAGM, these documents neither establish 
lines of reporting nor detail the responsibilities of management for 
monitoring product development. Furthermore, while the NCEH/ATSDR 
Policy: Clearance of Information Products, which guides ATSDR's review 
and clearance process, states that before a product is submitted for 
clearance immediate supervisors should ensure that the product is 
based on sound, ethical science and ensure the quality of the product, 
the policy provides no further guidance to immediate supervisors on 
carrying out these responsibilities. Because there is an absence of 
clearly defined lines of reporting and roles and responsibilities of 
management in these documents, management at various levels may not 
understand their specific responsibilities for overseeing product 
development. 

Although ATSDR's policies and procedures include some routine 
monitoring of the development of products produced by both ATSDR staff 
and staff of cooperative agreement partners, they do not consistently 
require that agency management monitor the development of key 
components of these products. The Standards states that internal 
control should be designed to ensure that ongoing monitoring occurs in 
the course of normal operations, and this includes management reviews 
of actual agency performance. ATSDR's policies and procedures require 
monitoring of key components of health studies, which use a detailed 
study protocol to guide a health study's development. ATSDR's Guidance 
for ATSDR Health Studies, which provides ATSDR staff with instructions 
on how to conduct a health study, states that a study protocol helps 
to ensure the quality of a health study and includes components such 
as a study's objectives, methodology, and timeline for completing key 
activities and milestones of a health study. At a minimum, if the 
study being conducted is deemed research, a study protocol is reviewed 
and approved within the appropriate division and may be sent out for 
scientific peer review before the health study begins. In addition, 
any health study involving human subjects must also be submitted to 
and approved by an established institutional review board.[Footnote 
39] The guidance explains that ongoing health study reviews are 
conducted to ensure that the study protocol is being followed, 
appropriate changes are made, the project remains on its established 
timeline, and enhancements to study quality are made when appropriate. 
Exposure investigations also use protocols that must be approved 
before the project is funded. These protocols include components such 
as a statement of the project's objectives as well as data analysis 
methods that will be used in completing the project. 

In contrast, ATSDR's policies and procedures for the development of 
public health assessments and health consultations, which are among 
the agency's core products, do not require management monitoring of 
key components of these products. For example, the PHAGM identifies an 
exposure assessment[Footnote 40] and health effects evaluation as the 
two primary technical components of the public health assessment 
process.[Footnote 41] Other components of the public health assessment 
process include data collection, community involvement, and 
development of conclusions and recommendations. However, there is no 
requirement that staff's work in any of these areas be reviewed and 
approved by management during the development of a product to ensure 
its accuracy and appropriateness. Furthermore, DHAC and DRO officials 
told us that there was no formal requirement for management to monitor 
or approve key components of public health products produced by their 
divisions, such as the product's methodology. When asked about 
monitoring requirements, a DRO official said that identifying the 
expertise needed for work at a site during the DHAC triage process 
helped to ensure that staff assigned to prepare a public health 
product had the skill sets required to make knowledgeable decisions on 
key components of a public health product. However, while identifying 
staff with the needed expertise to develop a public health product at 
initiation is beneficial, it is not a substitute for ongoing 
monitoring, which allows problems to be identified and addressed if 
they occur during a product's development. Further, during our small 
group interviews one DHAC employee expressed concern that because 
there were cases where only one person was developing a product, there 
would be no one to monitor that work until the product was submitted 
for review and clearance. 

Although ATSDR's policies and procedures for the development of public 
health assessments and health consultations do not require management 
monitoring of key components of these products, ATSDR officials said 
they held routine meetings during which issues specific to product 
development could be discussed. According to ATSDR officials, division 
directors schedule routine meetings with branch chiefs, and other team 
and site meetings are held. For example, according to ATSDR officials, 
DHAC and DHS officials meet with their respective division branch 
chiefs at least once each week to discuss projects and collaborate on 
site activities. However, none of the routine meetings described have 
established requirements for monitoring the development of key 
components of public health products. Additionally, weekly Issues 
Management and Senior Staff meetings, which are attended by senior 
division management, are used to discuss the work conducted at sites. 
ATSDR officials said that product development may be monitored during 
these meetings. However, these meetings rely on division management to 
bring problems or concerns regarding product development to the 
attention of the OD, and according to ATSDR officials, Issues 
Management meetings focus only on "sites of interest." Thus, while 
products related to "sites of interest" may be discussed at these 
meetings, current ATSDR procedures do not ensure the discussion of key 
components of products for ATSDR sites not identified as "sites of 
interest." Additionally, items on the agenda of the Issues Management 
meeting are not prioritized to ensure that the most significant 
problems associated with the development of a public health assessment 
or health consultation are promptly addressed. 

Because ATSDR's policies and procedures do not describe management's 
role for ensuring consistent monitoring of key product components, 
problems occurring during the development of ATSDR public health 
products may not be identified or addressed by management until the 
review and clearance phase, if at all. For example, in December 2001, 
the International Joint Commission requested ATSDR's assistance in 
evaluating the public health implications of the presence of hazardous 
materials in the Great Lakes region. According to ATSDR and Institute 
of Medicine reports, problems with ATSDR's Great Lakes report were not 
identified by management until the first draft of the document was 
completed in April 2004. Due to scientific concerns identified in the 
document once review and clearance began, the document underwent 
several years of reviews and revisions, and a final report was not 
issued until December 2008.[Footnote 42] 

ATSDR's Review and Clearance Policies and Procedures Do Not Always 
Reflect Current Practices and Do Not Establish a Process for Ensuring 
Consistent Review of All Products: 

While ATSDR has implemented policies and procedures governing the 
review and clearance of its public health products, some sections of 
ATSDR's review and clearance policies and procedures do not reflect 
current practices. The Standards for Internal Control in the Federal 
Government states that management is responsible for developing 
detailed policies, procedures, and practices to fit their agency's 
operations and ensuring that they are built into and become an 
integral part of operations. Additionally, the Internal Control 
Management and Evaluation Tool calls for policies and procedures to be 
regularly evaluated to ensure that they are still appropriate and 
working as intended. ATSDR uses the NCEH/ATSDR Policy: Clearance of 
Information Products to guide the review and clearance process. 
[Footnote 43] The clearance policy includes the NCEH/ATSDR Clearance 
Quick-Reference Guide, which outlines the required levels of review 
and clearance for each type of public health product.[Footnote 44] The 
clearance policy states that public health products may undergo 
required or discretionary review. The policy requires that all public 
health products be cleared through the initiating division,[Footnote 
45] and many public health products require additional review, such as 
review by the Office of Science. Some public health products may also 
undergo additional discretionary review when the originating division 
believes that a division outside of the required review process should 
be consulted. However, some sections of the NCEH/ATSDR Policy: 
Clearance of Information Products do not reflect current ATSDR 
practices. For example, the policy "highly recommends" that all public 
health products be reviewed and cleared by at least four individuals: 
the immediate supervisor, the branch chief, the associate director for 
science, and the division director.[Footnote 46] In addition, the 
policy's NCEH/ATSDR Clearance Quick-Reference Guide indicates that all 
public health assessments, health consultations, and exposure 
investigations must be reviewed and cleared by the division director 
or the division associate director for science. Yet according to DHAC 
management and staff, the review and clearance of DHAC products 
usually stops after review by branch chiefs within the division. 
[Footnote 47] Additionally, because the NCEH/ATSDR Clearance Quick-
Reference Guide is several years old, it does not describe the review 
and clearance requirements for new types of agency products such as 
"letter health consultations," which agency officials described as an 
expedited version of a health consultation.[Footnote 48] Also, the 
NCEH/ATSDR Clearance Quick-Reference Guide indicates that public 
health assessments, health consultations, and exposure investigations 
at "high priority" sites or "focus sites" must receive additional 
levels of review, but, as noted above, the agency no longer uses these 
designations. 

As of February 2010, the NCEH/ATSDR Policy: Clearance of Information 
Products also did not reflect current practices because it did not 
direct staff to use a CDC-required electronic clearance system called 
Documentum.[Footnote 49] The current clearance policy was effective in 
March 2006, prior to implementation of Documentum. Documentum is an 
electronic tool used by ATSDR to route public health products to the 
appropriate staff for review and clearance and to track the progress 
of each product during the process. In November 2009, officials told 
us that the agency planned to issue a revised clearance policy by the 
end of 2009. Additionally, although ATSDR officials said that staff 
should be having their products electronically reviewed and cleared 
through Documentum, management and staff told us during interviews 
that not all documents were being cleared using this system. Instead, 
documents that were not being entered into Documentum were being 
reviewed and cleared using a manual version of the review and 
clearance process. In November 2009, one ATSDR official estimated that 
only about 20 percent of DHAC documents, which include public health 
assessments, health consultations, and exposure investigations, were 
cleared using Documentum.[Footnote 50] However, per CDC policy, as of 
January 2010 all CDC centers, including ATSDR, were required to use 
Documentum to review and clear all agency products. In February 2010, 
ATSDR officials stated that all ATSDR divisions were currently using 
Documentum for the review and clearance of all documents but that the 
agency was still working to revise and update the clearance policy. 
Because some sections of ATSDR's review and clearance policies and 
procedures do not reflect current agency practices, staff cannot rely 
on them to accurately and consistently determine what review and 
clearance procedures to follow. Additionally, because there has not 
been uniform compliance with using Documentum, agency officials have 
been limited in their ability to track the review and clearance 
history for all of the agency's products and to ensure that the 
appropriate level of review was being conducted. 

In addition to not reflecting current practices, ATSDR's policies and 
procedures governing product review and clearance do not establish a 
process for ensuring that all products consistently receive 
appropriate review. The agency's clearance policy and procedures 
generally direct management and staff to use discretion to identify 
products that warrant a higher level of review, rather than 
determining review and clearance levels through a risk assessment 
process. As stated above, the Standards for Internal Control in the 
Federal Government states that effective internal control should 
provide for an assessment of the risks the agency faces, and that 
management needs to comprehensively identify risks and consider all 
significant interactions between the agency and other parties. 
However, the level of review and clearance that ATSDR products undergo 
varies by product type, rather than being determined by a 
comprehensive risk assessment of that particular product or site. For 
example, health study reports prepared by ATSDR staff are required to 
be reviewed and cleared by the originating division and ATSDR's Office 
of Science, and some must also undergo external peer review. In 
contrast, most public health assessments, health consultations, and 
exposure investigations are not required to be reviewed and cleared by 
ATSDR management any higher than DHAC branch chiefs.[Footnote 51] 
Based on the discretion of management and staff, some public health 
assessments, health consultations, and exposure investigations may 
also be submitted for additional review if they meet certain criteria. 
According to the ATSDR clearance policy, discretionary review is 
warranted when management or staff determines that a document (1) 
contains new or revised ATSDR policy (2) could have a high degree of 
visibility or (3) contains highly sensitive information. In addition, 
the DHAC Director has issued his own informal criteria to indicate 
which public health assessments and health consultations should 
undergo additional review beyond the branch chief level.[Footnote 52] 
However, even though ATSDR and DHAC have established criteria, there 
is no required point during a product's preparation where management 
and staff collectively determine whether a product meets the criteria, 
and if additional review is warranted. Because ATSDR does not conduct 
a comprehensive risk assessment of its products or sites, and its 
policies and procedures instead rely on management and staff 
discretion to make these determinations, the agency cannot ensure its 
products consistently receive the appropriate level of review and 
clearance.[Footnote 53] 

Management and staff discretion is also required in determining 
whether a public health assessment or health consultation should be 
submitted for external peer review. The ATSDR Peer Review Policy 
describes which public health products require external peer review, 
and states that all studies, results, or research that ATSDR carries 
out or funds in whole or in part must be peer reviewed. However, the 
policy specifically identifies public health assessments as one of the 
products that ATSDR does not consider "studies, results, or research." 
[Footnote 54] Because public health assessments are not required to 
undergo external peer review, ATSDR officials told us that management 
or staff could use their discretion to determine that a public health 
assessment or health consultation should be submitted for external 
peer review.[Footnote 55] According to ATSDR data, only 2 of the 282 
public health assessments and health consultations that were published 
in fiscal year 2008 underwent external peer review.[Footnote 56], 
[Footnote 57] 

During the March 2009 hearing before the House Committee on Science 
and Technology's Subcommittee on Investigations and Oversight, two 
participants suggested that ATSDR's public health assessments and 
health consultations should be required to undergo external peer 
review as a way to help ensure their quality.[Footnote 58] A 2000 
National Research Council report about peer review practices at EPA 
noted that peer review could promote efficiency if conducted in the 
early stages of a product's development, as well as assess and 
potentially improve the end products of scientific work.[Footnote 59] 
However, the report also noted that peer review had limitations, in 
that peer review could not substitute for technically competent work 
in the development of a product and could not ensure that regulatory 
policies and actions would be based on good science. 

Regional employees, ATSDR team leads, and nonmanagement employees in 
ATSDR Headquarters expressed mixed opinions to us about the use of 
external peer review for ATSDR public health products. In responses to 
a short questionnaire we administered during interviews with team 
leads and nonmanagement employees, 80 percent (24 of 30) said that 
external peer review would be either beneficial or sometimes 
beneficial in ensuring the quality of ATSDR public health products. 
Some of these employees reported that using external peer review may 
increase perceptions of the objectivity, credibility, and strength of 
their public health products. With regard to limitations, some 
employees reported that external peer review could cause further 
delays in the review and clearance process. Similarly, others 
suggested that external peer review should be conducted only for very 
complicated public health products or products with high levels of 
community concern or congressional interest. 

Conclusions: 

While administrative and management controls cannot guarantee product 
quality, they can help ensure the development of timely and credible 
public health products at ATSDR. And although ATSDR has established 
some policies and procedures to govern the preparation of its public 
health products, it lacks some critical controls to provide reasonable 
assurance of product quality, particularly for public health 
assessments, health consultations, and exposure investigations. The 
controls that have not been incorporated involve information flow, 
risk assessment, management roles and responsibilities, and monitoring. 

The lack of an agencywide product tracking system at ATSDR has 
hindered the effective flow of information about public health 
products between all levels of staff and management. It has also 
limited management's ability to monitor agency work and ensure that 
resources are being allocated appropriately, placing the OD in a 
reactive rather than leadership position with respect to the divisions 
and the public health work it manages. Once the Sequoia system becomes 
fully operational, management and staff should have a greater ability 
to obtain and share information about the agency's site-specific work, 
but it is too soon to determine whether they will take full advantage 
of Sequoia's capabilities. Furthermore, once implemented, those 
capabilities require that staff and cooperative agreement partners 
input data into the system as was intended. 

Additionally, without conducting risk assessments for the work being 
undertaken by the agency and using those risk assessments to guide 
agency processes for public health product preparation, ATSDR cannot 
provide reasonable assurance that its products have undergone the 
appropriate level of monitoring and review. If established, a risk 
assessment process could be used to determine the proper level of 
scrutiny throughout the initiation, development, and review and 
clearance phases, including whether or not a product should undergo 
external peer review, thereby ensuring that this determination is made 
consistently across the agency. Basing this process on a set of 
criteria, and documenting and tracking risk assessment decisions in 
agency systems, should help ensure an effective process. ATSDR has 
already incorporated some of the elements of risk assessment in the 
existing DHAC triage process for categorizing the priority of work 
requests. 

Finally, because the agency's policies lack guidance for management 
about their role in monitoring product development, ATSDR cannot be 
sure that management has a clear understanding of the role they are 
supposed to play in supervising a product's preparation. Additionally, 
ATSDR's policies and procedures for the development of public health 
assessments and health consultations do not require management's 
monitoring and approval of key components of a product during its 
development. Without adequate monitoring by management during a 
product's development, product errors may not be caught or significant 
publication delays may occur during the review and clearance phase, 
potentially undermining public confidence in the agency's products. 

Policies and procedures alone, however, cannot ensure the quality of 
ATSDR's public health products and, as noted above, internal controls 
provide only reasonable assurance, not absolute assurance, that an 
agency's objectives are being achieved. Issues outside of the 
influence of management, such as human mistakes, judgment errors, or 
acts by employees to circumvent management control, could also affect 
ATSDR's product quality. Nonetheless, improving ATSDR's policies and 
procedures regarding public health product preparation would help the 
agency provide greater assurance to those inside and outside the 
agency of the quality of these products. 

Recommendations for Executive Action: 

To strengthen ATSDR's policies and procedures, and ensure that they 
provide reasonable assurance of public health product quality, we 
recommend that the director of ATSDR take the following two actions: 

* Develop policies and procedures to ensure that a risk assessment is 
conducted at the time site-specific work is initiated, and that any 
assigned risk level be reevaluated throughout product preparation to 
ensure that it remains appropriate. 

* Revise existing policies and procedures, or develop new guidance, to 
provide documented direction for various levels of management on their 
roles and responsibilities in the monitoring of all products prior to 
review and clearance, such as requirements for management monitoring 
and approval of key components of these products. 

Agency Comments and Our Evaluation: 

ATSDR reviewed a draft of this report and provided written comments, 
which appear in appendix I. While ATSDR neither agreed nor disagreed 
with our recommendations and did not address them directly, in its 
comments ATSDR stated that the agency has begun to incorporate our 
recommendations. 

Although ATSDR did not comment directly on our recommendation that the 
agency conduct a risk assessment at the time site-specific work is 
initiated and reevaluate the assessment throughout product 
preparation, in its comments ATSDR stated that senior management was 
looking into formalizing and unifying coordination, triage, and 
prioritization of all incoming requests across the agency. ATSDR also 
acknowledged a need to make its prioritization process more explicit 
throughout the agency. It is imperative that ASTDR formalize its 
processes agencywide and ensure that its processes include a risk 
assessment to determine the proper level of scrutiny a product should 
receive throughout its preparation, including whether or not it should 
undergo external peer review. 

Related to our recommendation that ATSDR revise or develop policies 
and procedures to include direction for management in monitoring 
products prior to review and clearance, ATSDR noted that its process 
to formalize and unify coordination, triage, and prioritization of all 
incoming requests was expected to include the specification of 
management and staff roles and responsibilities from initiation 
through publication. It is important that ATSDR take this step in 
order to help ensure that management has a clear understanding of 
their responsibilities in supervising a product's preparation. 

In its comments, ATSDR noted that multiple guidelines are used to 
conduct its work and it uses an issues management process for agency 
risk management. Our findings document these guidelines and the issues 
management process, and describe their limitations in establishing 
effective information flow among all levels of management and staff, 
in providing a comprehensive assessment and categorization of the risk 
of work being initiated at the agency, and in identifying the roles 
and responsibilities of management. 

ATSDR also acknowledged that it would benefit from formalizing 
additional internal controls, and stated that as part of its review of 
the agency clearance policy it was incorporating a way to sample 
documents that were previously cleared to ensure that scientific 
principles are being applied across all divisions. ATSDR also stated 
that it expected Sequoia, its agencywide electronic project tracking 
system, to be fully implemented by the end of the year. 

Finally, ATSDR commented that we did not assess public comment as a 
part of our report, which it indicated was a critical component of the 
agency's quality assurance process. While we agree that public comment 
provides valuable input on those products which are subject to a 
public comment period, it augments but does not substitute for 
thorough internal review of a product or formal, external peer review 
of a product by carefully selected experts. 

ATSDR also provided technical comments, which we incorporated as 
appropriate. 

As we agreed with your offices, unless you publicly announce the 
contents of this report earlier, we plan no further distribution of it 
until 30 days from the report date. At that time, we will send copies 
to the Secretary of Health and Human Services and other interested 
parties. The report also will be available at no charge on GAO's Web 
site at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions about this report, please 
contact me at (202) 512-7114 or bascettac@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix II. 

Signed by: 

Cynthia A. Bascetta: 
Director, Health Care: 

[End of section] 

Appendix I: Comments from the Agency for Toxic Substances and Disease 
Registry: 

Department Of Health & Human Services: 
Office Of The Secretary: 
Assistant Secretary for Legislation: 
Washington, DC 20201: 

April 7, 2010: 

Cynthia A. Bascetta: 
Director, Health Care: 
U.S. Government Accountability Office: 
441 G Street N.W. 
Washington, DC 20548: 

Dear Ms. Bascetta: 

Enclosed are comments on the U.S. Government Accountability Office's 
(GAO) report entitled: "Agency For Toxic Substances And Disease 
Registry: Policies and Procedures for Public Health Product 
Preparation Should Be Strengthened" (GA0-10-449). 

The Department appreciates the opportunity to review this report 
before its publication. 

Sincerely, 

Signed by: 

Andrea Palm: 
Acting Assistant Secretary for Legislation: 

Enclosure: 

[End of letter] 

General Comments of the Department of Health and Human Services (HHS) 
on the Government Accountability Office's (GAO) Draft Report Entitled: 
Agency for Toxic Substances and Disease Registry: Policies and 
Procedures for Public Health Product Preparation Should be 
Strengthened 'GAO-10-449): 

The Agency for Toxic Substances and Disease Registry (ATSDR) wishes to 
thank GAO for the opportunity to review and comment on this Draft 
Report. ATSDR conducts valuable work and we appreciate GAO calling 
attention to areas where we can improve the documentation and 
functioning of our processes and controls. 

As stated in the Report, GAO "did not evaluate ATSDR's policies and 
procedures on scientific and technical risk assessment...did not 
review ATSDR products to assess their quality,...[and does] not 
express any view about their accuracy, completeness, or scientific 
credibility." However, GAO concluded that the "policies and procedures 
that ATSDR has established for public health product preparation lack 
some of the critical controls to provide reasonable assurance of 
product quality". 

In practice, ATSDR has a number of guidelines that are being utilized 
regarding site work, document preparation and clearance, and 
scientific practices. ATSDR provided GAO with twenty-five guidance 
documents that are currently employed, including the Public Health 
Assessment Guidance Manual, SOPs for Site-Specific Team Work, Guidance 
for Conducting a Review of Health Outcome Data, the Check List for the 
Exposure Investigation Process, and the NCEH/ATSDR Clearance Policy. 
In addition, as of January 1, 2010, all documents must be cleared 
through the e-clearance system, which documents proper clearance by 
the appropriate officials, and provides reporting that can be used by 
management. 

Currently, ATSDR ensures the proper handling of projects by discussing 
resources needed and justification of the allocation of resources for 
projects within and among ATSDR divisions. Each division has criteria 
for prioritizing work and identifying potential sites or items of 
interest; agency-wide risk management is handled through an issues 
management process, which includes weekly meetings with senior staff. 

GAO has not assessed public comment as a part of this report, which is 
a very critical component of ATSDR's quality assurance process. 
ATSDR's policies and procedures require public comment on all public 
health assessments, as well as many of our other documents. Through 
this process, ATSDR gains valuable comments from all affected parties, 
including citizens, academic experts, and industry experts. 

Nonetheless, ATSDR recognizes that we would benefit from formalizing 
additional internal controls. NCEH/ATSDR is currently working on 
revised clearance guidelines, which will incorporate a way to sample 
documents that have been cleared at the division level, in order to 
ensure that consistent scientific principles are being applied across 
all divisions. We also expect to fully implement the new agency-wide 
Sequoia system by the end of the year, which will track projects and 
compile scientific data in a centralized location. New projects and 
workplans are already being entered into the Sequoia system. 

Senior management is also looking into formalizing and unifying 
coordination, triage, and prioritization of all incoming requests 
across the agency. This process is expected to include project 
management and information flow, specify management and staff roles 
and responsibilities from initiation through publication, and inform 
the review and clearance process. 

ATSDR expends significant time and attention on our priority sites and 
work products. We acknowledge a need to make our prioritization 
process more explicit throughout the agency, balancing process with 
the timely production of quality products. 

ATSDR does important and dependable work in a challenging environment, 
and we continually incorporate improvements to the way in which we 
conduct our work. ATSDR has already begun to incorporate the 
recommendations, and we will continue to do so as we move forward. 

[End of section] 

Appendix II: GAO Contact and Staff Acknowledgments: 

Contact: 

Cynthia A. Bascetta at (202) 512-7114 or bascettac@gao.gov: 

Acknowledgments: 

In addition to the contact named above, key contributors to this 
report were Karen Doran, Assistant Director; George Bogart; Amy C. 
Leone; Roseanne Price; Mario D. Ramsey; Christina Ritchie; and Carla 
Willis. 

[End of section] 

Footnotes: 

[1] ATSDR was established within the Public Health Service of the 
Department of Health and Human Services. ATSDR is supported by the 
Centers for Disease Control and Prevention (CDC) and located within 
CDC's Office of Noncommunicable Diseases, Injury, and Environmental 
Health. 

[2] ATSDR is also responsible for educating the public and health care 
professionals regarding contaminant exposures and for establishing 
disease registries. However, these responsibilities are not the 
subject of this report. 

[3] GAO, Superfund: Public Health Assessments Incomplete and of 
Questionable Value, [hyperlink, 
http://www.gao.gov/products/GAO/RCED-91-178] (Washington, D.C.: Aug. 
1, 1991). In May 2007, we issued a report that included an expert 
panel evaluation of the design of one ATSDR health study. We reported 
that the expert panel found that many parameters of that study were 
appropriate, but that some experts suggested potential modifications 
to the study. See GAO, Defense Health Care: Activities Related to Past 
Drinking Water Contamination at Marine Corps Base Camp Lejeune, 
[hyperlink, http://www.gao.gov/products/GAO-07-276] (Washington, D.C.: 
May 11, 2007). 

[4] Institute of Medicine, Review of ATSDR's Great Lakes Report Drafts 
(Letter Report) (Washington, D.C.: National Academies Press, 2008). 

[5] ATSDR's Board of Scientific Counselors is an advisory committee 
that provides advice and guidance to the ATSDR Director. At ATSDR's 
request, the Board of Scientific Counselors convened a work group to 
evaluate the agency's peer review processes. The board issued a report 
in March 2009; as of March 2, 2010, the report was not available on 
ATSDR's Web site. 

[6] Agency for Toxic Substances and Disease Registry, Health 
Consultation: Formaldehyde Sampling at FEMA Temporary Housing Units, 
Baton Rouge, Louisiana (Atlanta, Ga.: February 1, 2007). 

[7] Majority Staff Report, Subcommittee on Investigations and 
Oversight, Committee on Science and Technology, U.S. House of 
Representatives, Toxic Trailers - Toxic Lethargy: How the Centers for 
Disease Control and Prevention Has Failed to Protect the Public Health 
(Washington, D.C.: September 2008). 

[8] As of January 15, 2010, the former ATSDR director took a new 
position at CDC, and an acting director has been appointed while a 
search for a permanent director is conducted. This former ATSDR 
director was the agency's director during the majority of the time our 
audit work was conducted. 

[9] Launched in 2009 by ATSDR and its companion organization--the 
National Center for Environmental Health--the National Conversation on 
Public Health and Chemical Exposures includes six work groups to 
research and make recommendations on cross-cutting public health and 
chemical exposure issues. Final work group reports will be submitted 
to a Leadership Council for inclusion in a final action agenda in the 
spring of 2011. 

[10] In this report we use the term preparation when referring 
collectively to the phases of initiation, development, and review and 
clearance of public health products. 

[11] In fiscal year 2008, ATSDR issued 60 public health assessments, 
222 health consultations, 10 exposure investigations, and 9 health 
study reports. 

[12] See GAO, Standards for Internal Control in the Federal 
Government, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: 
November 1999). Internal control is synonymous with management control 
and comprises the plans, methods, and procedures used to meet 
missions, goals, and objectives. 

[13] The Office of Management and Budget's (OMB) Circular No. A-123 
also defines management's responsibility for internal control in 
federal agencies. The internal control standards and the definition of 
internal control used in this circular are based on GAO's Standards 
for Internal Control in the Federal Government. See OMB Circular No. A-
123, (Revised): Management's Responsibility for Internal Control (Dec. 
21, 2004). 

[14] The Internal Control Management and Evaluation Tool is based on 
the Standards for Internal Control in the Federal Government, and it 
is intended to provide a systematic approach to assessing an agency's 
internal control structure. It is one in a series of related documents 
we have issued to assist agencies in improving or maintaining 
effective operations. See GAO, Internal Control Management and 
Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] 
(Washington, D.C.: August 2001). 

[15] ATSDR team leads are located in various ATSDR divisions and can 
have supervisory responsibilities, including assigning and planning 
work for staff, and monitoring and reporting on work progress to 
management. 

[16] ATSDR cooperative agreement partners are state agencies and one 
tribal government that ATSDR provides with funding and technical 
support to assess environmental health concerns at sites within their 
jurisdiction and to conduct or coordinate appropriate public health 
interventions. Cooperative agreement partners prepare public health 
products that are monitored, reviewed, and cleared by ATSDR. 

[17] We excluded employees with purely administrative responsibilities 
from the population of nonmanagement employees in each division. 

[18] We interviewed all employees in these offices who were available 
to participate on the day of the interview. 

[19] Pub. L. No. 96-510, 94 Stat. 2767. Under this law, EPA has 
responsibility to clean up highly contaminated waste sites and address 
the threats that these sites pose to human health and the environment. 

[20] Pub. L. No. 96-510, § 104(i), 94 Stat. 2778-2779. 

[21] NPL is a list of seriously contaminated hazardous waste sites 
that have been identified by the Superfund program. 

[22] Pub. L. No. 99-499, § 110, 100 Stat. 1613, 1636-1642. SARA 
requires that public health assessments include preliminary 
assessments of potential risk to human health based on such factors as 
the nature and extent of site contamination, the potential pathways of 
human exposure, the size and susceptibility of the community, and the 
effects of exposure associated with identified hazardous substances. 
SARA lists two purposes for health assessments--helping to decide 
whether (1) actions should be taken to reduce human exposure to a 
site's hazardous substances, and (2) additional information on human 
exposure and associated health risks is needed and should be acquired. 

[23] According to ATSDR, exposure investigations are not generalizable 
beyond the population studied. 

[24] ATSDR and NCEH are collectively known as NCEH/ATSDR. While these 
organizations share the same OD, they perform different functions. Our 
review focused only on those public health products produced by ATSDR. 

[25] Cross-clearance involves review and clearance of public health 
products by other persons or divisions that may have been involved in 
the production of the public health product, such as providing data or 
having staff serve as coauthors, or that may be affected by the 
product's content. Cross-clearance is conducted both within NCEH/ATSDR 
and across other centers at CDC. 

[26] The Board of Scientific Counselors, an advisory committee 
chartered under the Federal Advisory Committee Act, also provides 
advice and assistance to the Secretary of HHS and the Director of CDC. 

[27] The opportunity for a state or tribe to become a partner under a 
cooperative agreement occurs only at the beginning of the funding 
cycle. During the funding cycle, current state and tribal partners are 
asked to submit continuation applications and are funded based on 
available funding. According to ATSDR officials, the next cooperative 
agreement funding cycle, which will begin April 1, 2011, will be 
adjusted to a 3-year period. 

[28] Toxicological profiles summarize, interpret, and evaluate 
available data and possible health effects of hazardous substances 
found at NPL sites, and substances that pose the most significant 
potential threat to human health as determined by ATSDR and EPA. These 
products are typically developed in 2-year cycles. 

[29] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[30] [hyperlink, http://www.gao.gov/products/GAO-01-1008G]. 

[31] ATSDR officials said that they took HazDat off line because it 
became outdated after CDC updated its own system and no longer 
provided support for HazDat. 

[32] The DHAC Tracking and Triage Database is used to track work 
requests assigned to DHAC staff, including public health assessments, 
health consultations, and exposure investigations authored by staff 
within DHAC and cooperative agreement partners. ATSDR officials said 
that petition requests from individuals or groups are evaluated by a 
Petition Coordinator, a Petition Evaluation Team, and the division 
director, and tracked in the Petition Database. 

[33] The three ATSDR employees that have access to the Petition 
Database are the Petition Coordinator, one public health analyst, and 
one administrative specialist within DHAC. 

[34] ATSDR stated that these databases are also used to satisfy annual 
reporting requirements to the Congress and the Office of Management 
and Budget. 

[35] In addition to providing information about newly initiated work, 
Sequoia will also be able to provide information about products in 
various stages of development, as well as products that have already 
been issued. 

[36] ATSDR officials stated that major system development of Sequoia 
was expected to be completed by September 2010, and data entry 
completed by December 2010. 

[37] The risk assessment process described here is a management 
control process and is distinct from and not related to the risk 
assessment process used by EPA at Superfund sites. EPA uses risk 
assessment to characterize the nature and magnitude of health risks to 
humans (e.g., residents, workers, recreational visitors) and 
ecological receptors (e.g., birds, fish, wildlife) from chemical 
contaminants and other stressors that may be present in the 
environment. 

[38] ATSDR has established five distinct conclusion categories, which 
are based on the level of public health hazard that a site or 
hazardous substance might pose. 

[39] Institutional review boards review and monitor human subjects 
research, with the intended purpose of protecting the rights and 
welfare of the research subjects. 

[40] An exposure assessment is the process of finding out how people 
come into contact with a hazardous substance, how much of the 
substance they are in contact with, and where the substance is 
located. An exposure assessment reviews data collected by other 
federal and state government agencies, and differs from an exposure 
investigation in which ATSDR staff collect and analyze site-specific 
environmental or biological samples to determine whether individuals 
have been exposed to hazardous substances. 

[41] The public health assessment process is the method that ATSDR 
uses to evaluate the public health implications of exposures to 
environmental contamination. While this process bears the name of an 
ATSDR product, the public health assessment process itself may lead to 
a variety of products, including the public health assessment and the 
health consultation. 

[42] See Institute of Medicine of the National Academies, Review of 
ATSDR's Great Lakes Report Drafts (Letter Report) (Washington, D.C.: 
2008); Agency for Toxic Substances and Disease Registry, ATSDR Studies 
on Chemical Releases in the Great Lakes Region (Atlanta, Ga.: 2008); 
and Agency for Toxic Substances and Disease Registry, Statement of 
Scientific Concerns About the Draft Report, Public Health Implications 
of Hazardous Substances in the Twenty-Six U.S. Great Lakes Areas of 
Concern (Atlanta, Ga.: 2008). 

[43] In its 2009 report, ATSDR's Board of Scientific Counselors 
concluded that ATSDR's peer review process generally achieved agency 
quality assurance goals, but identified six general areas of concern 
and provided recommendations to address those concerns. Where the work 
group had concerns or recommendations relevant to our findings, we 
have included that information in this report. 

[44] The ATSDR Board of Scientific Counselors report stated that this 
guide was unnecessarily complex, could be simplified, and there was 
evidence that not all branch managers knew about the guide or paid 
attention to it. 

[45] Public health products that involve coauthors from another 
division or office; include content that directly pertains to relevant 
policy in another division or office; include comments on the program 
areas of another division or office; or include data collected and 
maintained by another division or office are also required to be 
reviewed and cleared by those divisions or offices. The clearance 
policy refers to this process as cross-clearance. 

[46] The policy states that, at a minimum, all public health products 
should be reviewed and cleared by the division director or designee. 

[47] Certain public health assessments and health consultations that 
meet specific criteria are also required to be reviewed by the 
associate director for science and the division director. 

[48] Agency officials stated that letter health consultations are 
subject to the same review and clearance requirements as health 
consultations. 

[49] The ATSDR Board of Scientific Counselors report also noted that 
the use of Documentum was not included in the clearance policy and 
stated that the policy should be updated. 

[50] After we discussed this discrepancy with ATSDR officials, the 
division director of DHAC issued a memorandum on November 20, 2009, 
directing all DHAC staff to use Documentum for the review and 
clearance of all DHAC documents. 

[51] ATSDR officials told us that protocols for conducting an exposure 
investigation are reviewed by the division associate director for 
science. 

[52] ATSDR officials told us that the DHAC associate director for 
science must review public health assessments and health consultations 
if they involve (1) a site that is categorized as an Urgent Public 
Health Hazard or a Health Advisory site; (2) a site where the "health 
call"--a determination of the health hazards present at the site--is 
based on new, unique, or unusual approaches; (3) a high profile site 
or site of interest; (4) a position that is in possible conflict with 
EPA or other agencies; or (5) sites that involve nonroutine analysis. 
The DHAC associate director for science is given the discretion to 
also forward these documents to the division director for additional 
review and clearance. 

[53] As mentioned above, ATSDR previously incorporated some of the 
principles of risk assessment when the agency officially classified 
hazardous chemical sites as "high priority" or "focus sites," thereby 
requiring products resulting from those sites to undergo specific 
levels of review. However, ATSDR no longer uses these designations. 

[54] This is consistent with SARA, which exempts health assessments 
from required peer review. Pub. L. No. 99-499, § 110, 100 Stat. 1641. 

[55] In commenting on ATSDR's external peer review policies, the ATSDR 
Board of Scientific Counselors' report stated that there should be a 
clear written policy on when external peer review is required and what 
it constitutes. 

[56] ATSDR employees told us that all products resulting from 
Department of Energy sites are submitted for external peer review. 
Both products that underwent external peer review in 2008 were public 
health assessments conducted at Department of Energy sites. In 
addition to the formal peer review completed for these two products, 
ATSDR reported that it also solicited informal comments from one or 
more subject-matter experts on four products before the products were 
finalized in 2008. 

[57] ATSDR officials told us that all public health assessments and 
some health consultations are also made available to the public for 
review and comment for 60 to 90 days. They stated that the agency 
reviews all public health comments and provides responses to them. 

[58] In 1991, we recommended that at least a sample of future ATSDR 
public health assessments undergo external peer review. However, as 
mentioned above, ATSDR does not currently have such a policy and 
instead relies on management and staff discretion to determine which 
public health assessments should be submitted for external peer review. 

[59] National Research Council, Strengthening Science at the U.S. 
Environmental Protection Agency: Research-Management and Peer-Review 
Practices (Washington, D.C.: National Academies Press, 2000). 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: