This is the accessible text file for GAO report number GAO-08-440
entitled 'Chemical Assessments: Low Productivity and New Interagency
Review Process Limit the Usefulness and Credibility of EPA's Integrated
Risk Information System' which was released on April 29, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Committee on Environment and Public Works, U.S.
Senate:
United States Government Accountability Office:
GAO:
March 2008:
Chemical Assessments:
Low Productivity and New Interagency Review Process Limit the
Usefulness and Credibility of EPA's Integrated Risk Information System:
GAO-08-440:
GAO Highlights:
Highlights of GAO-08-440, a report to the Chairman, Committee on
Environment and Public Works, U.S. Senate.
Why GAO Did This Study:
The Environmental Protection Agency’s (EPA) Integrated Risk Information
System (IRIS) contains EPA’s scientific position on the potential human
health effects of exposure to more than 540 chemicals. IRIS is a
critical component of EPA’s capacity to support scientifically sound
environmental decisions, policies, and regulations. GAO was asked to
examine (1) the outcome of steps EPA has taken to ensure that IRIS
contains current, credible chemical risk information, to address the
backlog of ongoing assessments, and to respond to new requirements from
the Office of Management and Budget (OMB); and (2) the potential
effects of planned changes to the IRIS assessment process on EPA’s
ability to ensure that IRIS provides current, credible risk
information. To do this work, GAO reviewed and analyzed EPA data and
interviewed officials at relevant agencies.
What GAO Found:
EPA’s actions since 2000 to ensure that IRIS contains current, credible
risk information, to address its backlog of 70 ongoing assessments, and
to respond to new OMB requirements—including increasing funding and
revising the assessment process—have not enabled EPA to routinely
complete credible IRIS assessments or decrease its backlog. Although in
fiscal years 2006 and 2007 EPA sent 32 assessments to OMB for the first
of three required external reviews, EPA finalized only 4 assessments
during this period. This low level of productivity jeopardizes the
viability of the IRIS database. Further, an EPA analysis indicated that
many existing assessments may need to be updated, and EPA program
offices and other IRIS users have requested assessments of hundreds of
chemicals not yet in IRIS. Factors contributing to EPA’s inability to
complete IRIS assessments in a timely manner include new OMB-required
reviews of IRIS assessments by OMB and other federal agencies; certain
EPA management decisions, such as delaying some assessments to await
new research; and the compounding effect of delays—even one delay can
have a domino effect, requiring the process to essentially be repeated
to incorporate changing science. As of December 2007, most of the 70
ongoing assessments had been in progress for over 5 years.
Regarding new OMB requirements, the IRIS assessment process now
includes two OMB/interagency reviews of draft assessments. These
reviews have resulted in involvement of other federal agencies in EPA’s
IRIS assessment process in a manner that limits the credibility of IRIS
assessments and hinders EPA’s ability to manage them. That is, the
OMB/interagency reviews lack transparency—OMB considers agencies’
comments on IRIS assessments to be internal executive branch documents
that may not be made public. Given the importance of IRIS assessments,
it is essential that input from all parties, including other federal
agencies, be part of the public record. Transparency is especially
important because agencies providing input include those that may be
affected by the assessments should they lead to regulatory or other
actions. Also, without communicating its rationale for doing so, OMB
directed EPA to terminate five assessments that for the first time
addressed acute, rather than chronic, exposure—even though EPA
initiated this type of assessment to help it implement the Clean Air
Act. Most OMB/interagency reviews completed to date have added 6 or
more months to the IRIS time frames.
Such delays and credibility concerns would likely be exacerbated by
further changes EPA is planning to respond to continuing concerns of
other federal agencies, such as providing them with an expanded role in
EPA’s IRIS assessment process and discretion to suspend assessments to
develop new studies for some chemicals. EPA estimates that such
assessments would take up to 6 years, an estimate GAO believes is
conservative in light of the assessment time frames under the current
process. Suspending assessments is inefficient; alternatively, with
longer-term planning, EPA could provide agencies and the public with
more advance notice of assessments, enabling them to complete relevant
research before IRIS assessments are started.
What GAO Recommends:
GAO recommends that EPA (1) clearly define and document an IRIS
assessment process that, among other things, can be conducted within a
time frame that minimizes the need for rework and (2) ensure that it
can develop transparent, credible assessments by, for example,
determining the types of IRIS assessments it will conduct based on EPA
program needs and defining the appropriate role of other federal
agencies in its IRIS assessment process. EPA agreed to consider GAO’s
recommendations in revising the IRIS assessment process.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-440]. For more
information, contact John B. Stephenson at (202) 512-3841 or
stephensonj@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
EPA's Efforts to Improve the IRIS Assessment Program Have Not Produced
the Desired Results:
While Appropriate Coordination with Federal Agencies Could Help EPA
Resolve IRIS Assessment Controversies More Efficiently, EPA's Proposed
Expansion of Agencies' Roles in IRIS Assessments Would Cause Further
Delays and Limit Their Credibility:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Broad Outline of EPA's Current IRIS Assessment Process:
Appendix III: Information on OMB/Interagency Reviews of IRIS
Assessments:
Appendix IV: Comments from the Environmental Protection Agency:
Appendix V: Comments from the Office of Management and Budget:
Appendix VI: GAO Contact and Staff Acknowledgments:
Figures:
Figure 1: National Academies' Risk Assessment and Risk Management Model
Used by EPA:
Figure 2: Funding for the IRIS Program, Fiscal Years 2000-2007:
Figure 3: Number of Completed IRIS Assessments, Draft Assessments to
OMB, and IRIS Staff in Full-Time Equivalents, Fiscal Years 2000-2007:
Figure 4: December 2007 Status of IRIS Assessments That EPA Reported as
Ongoing in Fiscal Year 2007:
Figure 5: EPA's Draft Proposed IRIS Assessment Process:
Figure 6: Status of IRIS Assessments Sent to OMB for the First OMB/
Interagency Review Starting in Fiscal Year 2004, as of December 1,
2007:
Abbreviations:
ATSDR: Agency for Toxic Substances and Disease Registry:
DOD: Department of Defense:
DOE: Department of Energy:
EPA: Environmental Protection Agency:
IARC: International Agency for Research on Cancer:
IRIS: Integrated Risk Information System:
NASA: National Aeronautics and Space Administration:
NCEA: National Center for Environmental Assessment:
NCI: National Cancer Institute:
NIOSH: National Institute of Occupational Safety and Health:
OMB: Office of Management and Budget:
ORD: Office of Research and Development:
PART: Program Assessment Rating Tool:
PBPK: physiologically based pharmacokinetic:
RfC: inhalation reference concentration:
RfD: oral reference dose:
TCE: trichloroethylene:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
March 7, 2008:
The Honorable Barbara Boxer:
Chairman:
Committee on Environment and Public Works:
United States Senate:
Dear Madam Chairman:
The Environmental Protection Agency's (EPA) Integrated Risk Information
System (IRIS)--a database integral to the agency's mission of
protecting human health and the environment--contains EPA's scientific
position on the potential human health effects that may result from
exposure to various chemicals in the environment. IRIS data provide the
fundamental scientific components needed to develop human health risk
assessments. These health risk assessments, in turn, provide the
foundation for risk management decisions, such as whether EPA should
establish air and water quality standards to protect the public from
exposure to toxic chemicals or set cleanup standards for hazardous
waste sites. In addition, state and local environmental programs, as
well as some international regulatory bodies, rely on IRIS health
effects information in managing their environmental protection
programs. Although the information in IRIS is a critical primary
component of EPA's capacity to support scientifically sound decisions,
policies, and regulations, many IRIS assessments are outdated, and few
assessments have been completed in recent years. This has resulted in a
significant backlog of incomplete chemical assessments and a growing
number of outdated assessments. Further, while EPA's IRIS database
currently includes about 540 chemicals, every year approximately 700
new chemicals enter commerce, any number of which could pose
significant human health risks.
Overall, the goal of the IRIS assessment process is to produce
quantitative estimates of cancer and noncancer effects from chronic
(long term) exposure to the chemicals assessed. One impact of not
having current and complete IRIS assessments of many potentially
harmful chemicals is that some chemicals that pose health risks to the
public may not be regulated under, for example, air or drinking water
statutes, or are regulated by standards that may not sufficiently take
into account the best available science on human health effects. For
example, trichloroethylene (TCE), a solvent widely used as a degreasing
agent in industrial and manufacturing settings, is the most frequently
reported organic contaminant in groundwater and has been linked to
cancer and other health hazards, according to the National
Academies.[Footnote 1] Yet, because of questions raised by peer
reviewers about the IRIS cancer assessment for TCE, EPA withdrew the
assessment from IRIS in 1989, did not initiate a new TCE assessment
until 1998, and likely will not complete that assessment until 2010 or
later. This delay represents an information gap of at least 21 years.
Without completed IRIS assessments reflecting current risk data, EPA
lacks assurance that its regulatory decisions concerning this
widespread chemical reflect the best available science on its potential
health effects.
While the IRIS assessment process includes numerous individual steps or
activities, major assessment steps include (1) a review of the
scientific literature; (2) preparation of a draft IRIS assessment; (3)
internal EPA reviews of draft assessments; (4) two Office of Management
and Budget (OMB)/interagency reviews, managed by OMB that provide for
input from OMB as well as from other federal agencies, including those
that may be affected by the IRIS assessments if they lead to regulatory
or other actions; (5) an independent peer review conducted by a panel
of experts; and (6) the completion of a final assessment that is posted
to the IRIS Web site. EPA's assessment process has undergone a number
of formal and informal changes during the past several years. The
agency is planning further changes--particularly in the areas of
external reviews and scientific data gaps--largely to address concerns
of other federal agencies, such as the Department of Defense (DOD).
Some of the assessment process changes have raised concerns about EPA's
ability to keep its scientific assessments separate from its risk
management decisions, as the National Academies recommends and EPA
policy endorses.
In this context, this report examines (1) the outcome of steps EPA has
taken to ensure that IRIS contains current, credible chemical risk
information, to address the backlog of IRIS assessments, and to respond
to new requirements from OMB; and (2) the potential effects of EPA's
planned changes to the IRIS assessment process on EPA's ability to
ensure that IRIS provides current, credible risk information.
In conducting our work, we obtained and analyzed information on EPA's
productivity, including the number of new and completed IRIS
assessments, for fiscal years 2000 through 2007; the status of IRIS
assessments, as of December 1, 2007, that were in progress during
fiscal year 2007; the status of IRIS assessments that have been sent to
OMB for OMB/interagency review; the number of assessments in the IRIS
database that may need to be updated; the resources provided to the
program for fiscal years 2000 through 2007; and user needs and EPA's
assessment completion goals. We interviewed EPA's National Center for
Environmental Assessment officials who manage the IRIS assessment
program, as well as officials from other EPA program offices and
federal science and health agencies that are involved in the IRIS
assessment process, to obtain their perspectives on, among other
things, the current IRIS assessment process, the potential effects of
the proposed changes to the process, the extent to which EPA has made
progress in completing assessments and meeting user needs, and
challenges EPA faces in completing assessments. In addition, we also
interviewed officials from the Department of Defense, the Department of
Energy (DOE), the National Aeronautics and Space Administration (NASA),
and OMB to obtain their perspectives on the OMB/interagency review
process and on the planned changes to the IRIS assessment process. We
did not evaluate the scientific content or quality of IRIS assessments.
(See app. I for a more detailed description of the methodology we
employed.) We conducted our work from October 2006 to March 2008 in
accordance with generally accepted government auditing standards, which
require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Results in Brief:
EPA has taken a number of steps to help ensure that IRIS contains
current, credible chemical risk information, to address its backlog of
ongoing assessments, and to respond to new OMB requirements. However,
to date, these changes--including increasing funding, centralizing
staff conducting assessments, and revising the assessment process--have
not enabled EPA to routinely complete credible IRIS assessments or
decrease the backlog. That is, although EPA sent 32 draft assessments
for external review in fiscal years 2006 and 2007, the agency finalized
only 4 IRIS assessments during this time. Several key factors have
contributed to EPA's inability to achieve a level of productivity that
is needed to sustain the IRIS program and database: new OMB-required
reviews of IRIS assessments by OMB and other federal agencies; the
growing complexity and scope of risk assessments; certain EPA
management decisions and issues, including delaying completion of some
assessments to await new research or to develop enhanced analyses of
uncertainty in the assessments; and the compounding effect of delays.
Regarding the last factor, even a single delay in the assessment
process can eventually lead to the need to repeat the assessment
process to take into account changes in science and methodologies. A
variety of delays have impacted the majority of the 70 assessments
being conducted as of December 2007--causing them to be in process for
more than 5 years. These time frames are problematic because of the
substantial rework such cases often require to take into account
changing science and methodologies before they can be completed.
Further, because EPA staff time continues to be dedicated to completing
these assessments, EPA's ability to both keep the more than 540
existing assessments up to date and initiate new assessments is
limited. Importantly, EPA program offices and state and local entities
have requested assessments of hundreds of chemicals not yet in IRIS,
and EPA data as of 2003 indicated that the assessments of 287 chemicals
in the database may be outdated--that is, new information could change
the risk estimates currently in IRIS or enable EPA to develop
additional risk estimates for chemicals in the database. In addition,
because EPA's 2003 data are now more than 4 years old, it is likely
that more assessments may be outdated now.
One of the factors that has contributed to EPA's inability to complete
assessments in a timely manner--the new OMB/interagency review process-
-also limits the credibility of the assessments because it lacks
transparency.[Footnote 2] Specifically, neither the comments nor the
changes EPA makes to the scientific IRIS assessments in response to the
comments made by OMB and other federal agencies, including those whose
workload and resource levels could be affected by the assessments, are
disclosed. According to OMB, the comments it provides to EPA and EPA's
disposition of them are considered internal executive branch
communications that may not be made public. Further, OMB has not
communicated its rationale for directing EPA to discontinue work on
five IRIS assessments that EPA had sent to OMB for OMB/interagency
review. These assessments, initiated to meet EPA program needs, were
the first EPA IRIS assessments of short-term (acute) risks of exposure;
the IRIS program historically has evaluated long-term (chronic) risks.
The additional assessment process changes EPA is planning would likely
exacerbate delays in completing IRIS assessments and further affect
their credibility. Specifically, despite the informal OMB/interagency
review process that OMB required EPA to incorporate into the IRIS
assessment process in 2005, certain federal agencies continue to
believe they should have greater and more formal roles in EPA's
development of IRIS assessments. Consequently, EPA has been working for
several years to establish a formal IRIS assessment process that would
expand the role of federal agencies in the process--including agencies
such as DOD, which could be affected by the outcome of IRIS
assessments. Some of these agencies and their contractors could face
increased cleanup costs and other legal liabilities if EPA issued an
IRIS assessment for a chemical that resulted in a decision to regulate
the chemical to protect the public. Under EPA's planned changes, these
potentially affected agencies would be able, at several points in the
assessment process, to subject particular chemicals of interest to
additional process steps. EPA estimates that these assessments would
take up to 6 years to complete because, for example, at the discretion
of these agencies, EPA would suspend the assessment process for up to
18 months so the agencies could conduct additional research to fill
data gaps, rather than proceeding with currently available data. While
it is important to ensure that assessments consider the best science,
EPA has acknowledged that waiting for new data can result in
substantial harm to human health, safety, and the environment. Further,
although coordination with other federal agencies about IRIS
assessments could enhance the quality of the assessments, increasing
the role of agencies that may be affected by IRIS assessments in the
process itself reduces the credibility of the assessments if that
expanded role is not transparent. In this regard, while EPA planned to
include federal agencies' comments in the public record, the process
changes have been delayed since early 2007 in part because of OMB's
view that agencies' comments about IRIS assessments represent internal
executive branch communications that may not be made public--a view
that is inconsistent with the principle of sound science that relies
on, among other things, transparency.
We are making recommendations to the EPA Administrator to require the
Office of Research and Development to re-evaluate its draft proposed
changes to the IRIS assessment process in light of the issues raised in
this report and ensure that any revised process, among other things,
clearly defines and documents an IRIS assessment process that will
enable the agency to develop the timely chemical risk information it
needs to effectively conduct its mission. In addition, we are
recommending that the EPA Administrator take steps to better ensure
that EPA has the ability to develop transparent, credible IRIS
assessments, including determining the types of assessments it needs to
support its programs; defining the appropriate role of external federal
agencies in EPA's IRIS assessment process; and managing an interagency
review process in a manner that enhances the quality, transparency,
timeliness, and credibility of IRIS assessments.
We provided EPA and OMB with a draft of this report for review and
comment. EPA agreed to consider our recommendations in revising the
IRIS assessment process. However, EPA stated that it believed the
productivity and transparency issues discussed in the draft report were
misrepresented in the title and body of the report. We disagree and
believe we have fairly represented IRIS productivity and transparency
issues related to the IRIS assessment process. We did, however, clarify
that the transparency issues highlighted in our report focus on the
IRIS assessment process rather than on the content of IRIS assessments,
and we revised the report title. In its comments, OMB did not
specifically address the recommendations we made to EPA but disagreed
with some aspects of the report, such as our characterization of the
purpose and effect of the OMB-managed interagency reviews of IRIS
assessments and our conclusion that interagency comments should be
transparent. We disagree with OMB and believe that we have fairly
represented the OMB/interagency review process as well as the
importance of making input from all parties publicly available to
alleviate concerns of potential bias. EPA's and OMB's letters and our
detailed responses are discussed further at the end of this report and
in appendixes IV and V.
Background:
EPA's Integrated Risk Information System (IRIS) is an important source
of information on health effects that may result from exposure to
chemicals in the environment. IRIS was created in 1985 to help EPA
develop consensus opinions within the agency about the health effects
from chronic exposure to chemicals, and its importance has increased
over time as EPA program offices and the states have increasingly
relied on IRIS information in making environmental protection
decisions. Today, the IRIS database--which currently contains
assessments of more than 540 chemicals--is heavily relied upon by EPA,
state and local environmental programs, international regulatory
bodies, academia, industry, and others to support risk-based decision
making to protect public health and the environment. According to EPA,
national and international users access the IRIS database approximately
9 million times a year. EPA's Assistant Administrator for the Office of
Research and Development has described IRIS as the premier national and
international source for qualitative and quantitative chemical risk
information; other federal agencies have noted that IRIS data is widely
accepted by all levels of government across the country for application
of public health policy, providing benefits such as uniform,
standardized methods for toxicology testing and risk assessment, as
well as uniform toxicity values. Similarly, a private-sector risk
assessment expert has stated that the IRIS database has become the most
important source of regulatory toxicity values for use across EPA's
programs and is also widely used across state programs and
internationally.
As shown in figure 1, the toxicity assessments in the IRIS database
fulfill the first two critical steps of the risk assessment process--
providing hazard identification and dose-response assessment. IRIS
information can then be used with the results of exposure assessments
(typically conducted by EPA's program or regional offices) to provide
an overall characterization of the public health risks for a given
chemical in a given situation. The risk characterization information
can be used to make risk management decisions designed to protect
public health. The development of risk assessments is thus directly
dependent on the development of toxicity assessments such as those
developed in the IRIS program.
Figure 1: National Academies' Risk Assessment and Risk Management Model
Used by EPA:
[See PDF for image]
This figure is an illustration of the National Academies' Risk
Assessment and Risk Management Model Used by EPA. The following
information is depicted:
Risk assessment:
* IRIS toxicity assessment:
1. Hazard identification;
2. Dose-response assessments.
* Exposure assessment;
* Together these assessments form:
Risk characterization.
Risk management:
* Development of regulatory options;
* Evaluation of public health, economic, social, and political
consequences of regulatory options;
* Added together with risk characterization, these yield:
* Agency decisions and actions.
Source: National Academies.
{End of figure]
Risk management, as opposed to risk assessment, involves integrating
the risk characterization information generated from the risk
assessment with other information, such as economic information on the
costs and benefits of mitigating the risk, technological information on
the feasibility of managing the risk, and the concerns of various
stakeholders, to decide how to protect public health. An initial risk
management decision would be to determine whether the health risks
identified in a chemical risk assessment warrant regulatory or other
actions. Examples of subsequent risk management decisions that could
stem from a determination that action is necessary to protect public
health include deciding (1) how much of a chemical a company may
discharge into a river; (2) which substances may be stored at a
hazardous waste disposal facility; (3) the extent to which a hazardous
waste site must be cleaned up; (4) permit conditions for treatment,
storage, or disposal of hazardous waste; (5) levels for air emissions;
and (6) allowable levels of contamination in drinking water. Thus, as
EPA has recognized, although IRIS assessments are not regulatory in
nature, the quantitative IRIS values may influence many environmental
decisions and may serve as a basis for regulatory consideration.
A typical IRIS assessment contains a qualitative hazard identification
description and quantitative dose-response assessments. Among other
things, a hazard identification description identifies the potential
noncancer and cancer health effects of exposure to a chemical that
research studies have suggested or determined. For example, for cancer
effects, EPA describes the potential health risk using one of five
weight-of-the-scientific-evidence descriptors, ranging from
"carcinogenic to humans" to "not likely to be carcinogenic to humans."
The quantitative assessments also address noncancer and cancer health
effects and are developed if there are sufficient credible research
data, primarily from either animal (toxicity) or human (epidemiology)
studies to support this type of analysis. The noncancer dose-response
assessments may include:
* an oral reference dose (RfD)--an estimate of the daily exposure to a
chemical that is likely to be without an appreciable risk of
deleterious effects during a person's lifetime--expressed in terms of
milligrams per kilogram, and:
* an inhalation reference concentration (RfC)--an estimate of the daily
exposure to a chemical that is likely to be without an appreciable risk
of deleterious effects during a person's lifetime--expressed in terms
of milligrams per cubic meter.
The quantitative cancer toxicity assessments include estimates of a
chemical's carcinogenic potency--a "cancer slope factor" and "unit
risks." Both the cancer slope factor and unit risks are estimates of
the increased cancer risk (e.g., 1 in 100 people getting cancer, 1 in
1,000, 1 in 1 million, etc.) from a lifetime of exposure to a given
chemical. However, the unit risk is an estimate of the increased risk
for lifetime exposure at a standard concentration of a chemical in air
or water (1 microgram per cubic meter of air or 1 microgram per liter
in water), whereas the cancer slope factor is an estimate of the
increased risk per unit dose (calculated using a dose-response curve,
or a graph that shows the relationship between a dose and the
proportion of exposed persons or animals that have a biologically
significant response).[Footnote 3]
Historically and currently, the focus of IRIS toxicity assessments has
been on the potential health effects of long-term (chronic) exposure to
chemicals. According to OMB,[Footnote 4] EPA is the only federal agency
that develops qualitative and quantitative assessments of both cancer
and noncancer risks of exposure to chemicals, and EPA does so largely
under the IRIS program. Other federal agencies develop qualitative
cancer assessments or quantitative estimates of noncancer effects of
exposure to chemicals in the environment. For example, the Agency for
Toxic Substances and Disease Registry (ATSDR) develops quantitative
estimates of the noncancer effects of exposures to chemicals in the
environment for exposures of up to 14 days (acute); more than 14 days
but less than a year (subchronic); and 365 days and longer (chronic).
While ATSDR's toxicological profiles include information from other
agencies' cancer assessments, including EPA's quantitative IRIS cancer
assessments, ATSDR does not develop quantitative cancer assessments.
ATSDR toxicological profiles also include qualitative cancer
assessments developed by the Department of Health and Human Services'
National Toxicology Program and the World Health Organization's
International Agency for Research on Cancer (IARC). While these latter
assessments provide information on the effects of long-term exposures
to chemicals, they only provide qualitative assessments of cancer risks
(e.g. known human carcinogen, likely human carcinogen, etc.) and not
quantitative estimates of cancer potency, which are required to conduct
quantitative risk assessments.
As the IRIS database became more widely used and accepted, EPA took
steps, beginning in the early 1990s, to improve and maintain the IRIS
program and database. For example, the agency created an IRIS Quality
Action Team that produced a report in 1994 outlining a number of
recommendations for improvement, and EPA implemented a pilot program
from 1995-1997 to test new operational procedures for the IRIS program.
Changes under the pilot program included developing a standard
toxicological review document to support each IRIS summary,
incorporating peer review into the assessment process, and establishing
a standing group of 18 senior health scientists to conduct the internal
agency review of all IRIS draft assessments. The standing group, now
called the IRIS Agency Review Committee, includes representatives from
the program offices and regions. EPA also formed an IRIS implementation
strategy team that developed recommendations in 1997 to improve the
IRIS program and database. Key recommendations addressed the need to
(1) update IRIS information, (2) establish an annual agenda for the
program, (3) form a central IRIS staff to be responsible for the
database and to coordinate with the program and regional offices
leading individual IRIS assessments, (4) provide Internet access to the
IRIS database, and (5) conduct more outreach to users.
In response, EPA formed a small centralized IRIS staff in the Office of
Research and Development, National Center for Environmental Assessment
(NCEA), which has implemented many of these recommendations. For
example, EPA placed the IRIS database on its Web page, set up a hot
line service to improve outreach with users, and started developing an
annual IRIS agenda that identifies the chemicals to be assessed during
the fiscal year (new and ongoing assessments) and providing this agenda
to the public in a notice in the Federal Register. EPA also responded
to the recommendations by posting external peer review drafts of IRIS
assessments on EPA's Web page and considering public comments received
on these drafts. As discussed below, EPA has continued to evaluate its
IRIS assessment process and make changes in an effort to improve it.
EPA's Efforts to Improve the IRIS Assessment Program Have Not Produced
the Desired Results:
In response to criticisms and in an effort to meet the needs of the
programs and entities that rely on IRIS information, EPA has continued
to make a number of IRIS assessment process changes aimed at improving
the timeliness, quality, consistency, and transparency of IRIS
assessments. However, even with process changes and increased program
funding and staffing, EPA has not been able to routinely complete
credible assessments or decrease its backlog of ongoing assessments.
Several key factors have contributed to EPA's inability to achieve a
level of productivity that is needed to sustain the IRIS program and
database, including the OMB/interagency review process managed by OMB,
certain management decisions and issues regarding the IRIS program, and
the compounding effect of delays. In addition, because the OMB/
interagency review process is not transparent, this change also limits
the credibility of IRIS assessments.
EPA Efforts to Improve IRIS Continue:
Despite the many steps that EPA took throughout the 1990s aimed at
improving the IRIS assessment process, the agency has continued to face
criticism that the risk information in the database is outdated and of
varying quality. For example, according to congressional testimony in
2000 by a risk assessment expert and a representative of a chemical
industry association, the outdated and inconsistent information in IRIS
represented a serious limitation that undermined the accuracy of risk
assessments and risk management decisions. In addition, external
parties--notably entities that may be affected by the IRIS assessments,
including other federal agencies and industry--have criticized the IRIS
assessment process as lacking transparency and have sought earlier
input into EPA's assessment process. Further, as a result of continuing
concerns that EPA and state regulators were relying on scientific
information that was potentially outdated, in 2000, a Senate
appropriations committee report directed EPA to conduct an assessment
to determine the need to both update and add new assessments to IRIS.
In response to the criticisms and in an effort to meet the needs of the
programs and entities that rely on IRIS information, EPA has continued
to make a number of IRIS assessment process changes aimed at improving
the timeliness, quality, consistency, and transparency of IRIS
assessments. For example, EPA further centralized the assessment
process, hiring additional scientists in the Office of Research and
Development to lead individual IRIS assessments; in the past, chemical
managers in various program offices and regions had led the assessments
on a voluntary basis. EPA made this change to improve the timeliness of
assessments and to address concerns about the inconsistency of
assessments that the different offices were producing. In addition, EPA
changed its peer review requirements, opting for peer review panels,
whose meetings the agency has opened to the public, rather than
obtaining written peer review comments from several experts. According
to EPA, these steps were taken to provide the best possible scientific
review of each assessment and to make the review process more
transparent.
EPA also decided to conduct its internal agency review of assessment
drafts earlier in the process. That is, the 18-member IRIS Agency
Review Committee comprising EPA senior health scientists now comments
on draft assessments before they are released for external review, as
well as after external review comments are incorporated in the
assessments. In the past, the review by the IRIS Agency Review
Committee had taken place once, following the external reviews.
According to EPA, this change was made to enhance peer reviews by
identifying key science issues and providing external reviewers with
drafts that had already been thoroughly vetted within EPA. Prior to
this change, IRIS assessments had been internally peer reviewed by
three or four scientists with relevant expertise in the Office of
Research and Development, program offices, or regions before the
assessments were sent to external peer review.[Footnote 5] EPA has also
added formal briefings of draft assessments to the Assistant
Administrator, Office of Research and Development, at various stages in
the assessment process. In addition, EPA informally elevated final IRIS
assessment approval authority to the Assistant Administrator, Office of
Research and Development. EPA has also delayed or suspended some
assessments to await new or scientific studies.
Another key change that EPA has incorporated into the IRIS assessment
process at OMB's request is an OMB/interagency review process that is
managed by OMB. The purpose of these reviews is to obtain input from
OMB and other federal agencies that OMB has determined have an interest
in particular IRIS assessments as they are being developed in order to
help ensure and increase their quality. The reviews occur at two points
in the IRIS assessment process--first, after the internal agency review
but before the external peer review; and second, after EPA has
incorporated input from the external peer review. According to EPA
officials, this OMB/interagency review process has evolved from an ad
hoc review of selected IRIS assessments of interest to OMB and other
federal agencies to a process that now requires, for all assessments,
OMB's determination that EPA has satisfactorily addressed all OMB/
interagency comments. This determination must be made both before EPA
can provide draft assessments to external peer reviews and before EPA
can finalize and post assessments on the IRIS database.
Although the IRIS assessment process continues to evolve, it currently
typically includes the following steps (see app. II for a flow chart of
the process):
* solicitation of chemical nominations from EPA program and regional
offices, federal agencies, and the public.
* selection of chemicals to be assessed during the fiscal year
(referred to as the annual agenda).
* publication of a Federal Register notice announcing EPA's annual
agenda--the specific chemical assessments the agency intends to
conduct--and requesting scientific information about these chemicals,
thereby giving the public and other federal agencies an opportunity to
identify relevant studies.
* a review of the scientific literature.
* development of draft IRIS toxicological reviews and summaries
containing qualitative and, if sufficient information is available,
quantitative risk estimates, that have undergone internal peer
consultation--a peer review by three or four EPA scientists with
relevant expertise.
* initial internal agency review, which includes review and comment
from the 18-member EPA IRIS Agency Review Committee.
* OMB/interagency review by other federal agencies, such as DOD, DOE,
and NASA, coordinated by the Office of Management and Budget; OMB
informs EPA when EPA has adequately addressed interagency comments.
* external peer review by a group of independent experts, convened by
an EPA contractor, the Science Advisory Board, or the National
Academies, and public comment.
* a second internal agency review.
* a second OMB/interagency review; OMB informs EPA when EPA has
adequately addressed interagency comments.
* EPA management review and approval.
* posting of IRIS assessments on the IRIS database, available on EPA's
Web site.
In addition to EPA continuing to revise its IRIS assessment process,
since fiscal year 2000, funding for the IRIS program has increased--
from $1.7 million to $9.6 million in fiscal year 2007 (see fig. 2). The
need for increased resources to accomplish significant IRIS
improvements had been noted in the February 1997 IRIS Implementation
Strategy Team Report.
Figure 2: Funding for the IRIS Program, Fiscal Years 2000-2007:
[See PDF for image]
This figure is a line graph depicting the following data:
Fiscal year: 2000;
Funding for the IRIS Program: $1.7 million.
Fiscal year: 2001;
Funding for the IRIS Program: $1.9 million.
Fiscal year: 2002;
Funding for the IRIS Program: $6.7 million.
Fiscal year: 2003;
Funding for the IRIS Program: $2.3 million.
Fiscal year: 2004;
Funding for the IRIS Program: $6.9 million.
Fiscal year: 2005;
Funding for the IRIS Program: $8.2 million.
Fiscal year: 2006;
Funding for the IRIS Program: $8.4 million.
Fiscal year: 2007;
Funding for the IRIS Program: $9.6 million.
Source: GAO analysis of EPA data.
Note: In fiscal year 2002, a congressional appropriations conference
committee designated $5 million to accelerate the development of new
IRIS values and to update current IRIS values. According to EPA
officials, this funding was provided to various EPA program offices to
support the IRIS assessments that program offices were leading at that
time. In addition, EPA has reprogrammed funds from some of its other
programs to expand the IRIS program to support the development of IRIS
assessments, especially high-priority chemicals.
[End of figure]
Process Changes and Increased Resources Have Not Enabled EPA to
Routinely Complete IRIS Assessments and Meet Users' Needs:
As shown in figure 3, EPA has continued to take steps to improve the
IRIS assessment process and increased program funding and staffing, but
the agency has not made progress in completing IRIS assessments and
reducing its backlog of ongoing assessments. Although the number of
program staff has quadrupled from 8 to 37 between 2000 and 2007, EPA
has, on average, completed about five IRIS assessments per year--and in
fiscal years 2006 and 2007, completed only two each year. However, EPA
sent 16 draft assessments to OMB for OMB/interagency review in both
fiscal years 2006 and 2007 and plans to provide 16 draft assessments to
OMB annually in fiscal years 2008 through 2012. Further, in its fiscal
year 2008 budget justification documents, EPA said it plans to complete
8 IRIS assessments in fiscal year 2008, noting that completion of
assessments, rather than providing drafts to OMB, is the most important
outcome.
Figure 3: Number of Completed IRIS Assessments, Draft Assessments to
OMB, and IRIS Staff in Full-Time Equivalents, Fiscal Years 2000-2007:
[See PDF for image]
This figure is a multiple line graph depicting the following data:
Fiscal year: 2000;
IRIS staff levels in full-time equivalents: 7.8;
IRIS assessments completed: 5.
Fiscal year: 2001;
IRIS staff levels in full-time equivalents: 7.8;
IRIS assessments completed: 7.
Fiscal year: 2002;
IRIS staff levels in full-time equivalents: 7.8;
IRIS assessments completed: 3.
Fiscal year: 2003;
IRIS staff levels in full-time equivalents: 12.8;
IRIS assessments completed: 11.
Fiscal year: 2004 (Year OMB/interagency review initiated);
IRIS staff levels in full-time equivalents: 27;
IRIS assessments completed: 4;
Draft IRIS assessments to OMB for OMB/interagency review: 1.
Fiscal year: 2005;
IRIS staff levels in full-time equivalents: 37;
IRIS assessments completed: 4;
Draft IRIS assessments to OMB for OMB/interagency review: 5.
Fiscal year: 2006;
IRIS staff levels in full-time equivalents: 37;
IRIS assessments completed: 2;
Draft IRIS assessments to OMB for OMB/interagency review: 16.
Fiscal year: 2007;
IRIS staff levels in full-time equivalents: 37;
IRIS assessments completed: 2.
Draft IRIS assessments to OMB for OMB/interagency review: 16.
Source: GAO analysis of EPA data.
[End of figure]
During fiscal year 2007, EPA had 77 ongoing IRIS assessments. As of
December 1, 2007, 70 of these were still in progress--2 were completed
in fiscal year 2007 and 5 were discontinued at the direction of
OMB.[Footnote 6] Representing EPA's first efforts to develop IRIS
assessments covering acute (short-term) exposure to chemicals, EPA
initiated them in 2004 to meet EPA program needs, including, for
example, to help it implement the Clean Air Act. In November 2007, EPA
officials told the members of the Board of Scientific Counselors
committee evaluating EPA's health risk assessment program that OMB did
not explain to EPA why it was directing the agency to terminate these
IRIS assessments.
Of the 70 assessments still in progress as of December 1, 2007, 48 have
been in progress for more than 5 years, and 12 of those for more than 9
years. As figure 4 shows, many of these assessments are either being
drafted or are undergoing internal agency review. Additionally, 19
assessments are undergoing one of the two OMB/interagency reviews
managed by OMB, and 5 of the assessments have been suspended by EPA for
various reasons, including waiting for new research studies that have
yet to be completed.
Figure 4: December 2007 Status of IRIS Assessments That EPA Reported as
Ongoing in Fiscal Year 2007:
[See PDF for image]
This figure is a vertical bar graph depicting the following data:
Type: Suspended[A];
Number of assessments: 5.
Type: Draft development[B];
Number of assessments: 31.
Type: Formal EPA agency review[C];
Number of assessments: 6.
Type: First interagency review managed by OMB[D];
Number of assessments: 14.
Type: Assessments discontinued per OMB direction[E];
Number of assessments: 5.
Type: Assessments delayed per EPA direction[F];
Number of assessments: 1.
Type: External peer review[G];
Number of assessments: 8.
Type: Second interagency review managed by OMB[H];
Number of assessments: 5.
Type: Preparation of final assessment/web posting[I];
Number of assessments: 0.
Type: Completed assessments[J];
Number of assessments: 2.
Source: GAO analysis of EPA information.
[A] EPA has suspended five IRIS assessments for several reasons,
including waiting for additional research, reevaluating the need for an
assessment, and potentially broadening the scope of an assessment.
[B] This phase includes conducting literature searches, drafting
assessments, and incorporating comments from an internal peer review
process; it is concluded when the IRIS Program Director provides the
draft and support documents for review by EPA's IRIS Agency Review
Committee.
[C] This phase includes the review by EPA's IRIS Agency Review
Committee and revisions in response to the review; it is concluded when
the NCEA Associate Director for Health transmits a draft to OMB.
[D] This phase encompasses the first OMB/interagency review managed by
OMB; revisions in response to the review; negotiations among EPA, OMB,
and other federal agencies over changes to the draft and the content of
the charge questions for the external peer reviewers; and briefing of
the Assistant Administrator for the Office of Research and Development
(ORD). It is concluded when OMB informs EPA that issues have been
resolved and the assessment can proceed to the next step--external peer
review.
[E] In November 2007, EPA said that in 2006 OMB requested and then in
2007 directed EPA to discontinue the five IRIS assessments covering
acute exposures that EPA had sent to OMB for OMB/interagency review
from July to September 2006.
[F] One assessment has been delayed by EPA for almost 2 years pending
the development of an acceptable uncertainty analysis.
[G] This phase encompasses public review and comment, the external peer
review conducted by an independent panel, and incorporation of draft
revisions in response to peer review and public comments. It is
concluded when the NCEA Associate Director for Health transmits a
revised draft to OMB for the second OMB/interagency review it manages.
[H] This phase encompasses the second review by EPA's IRIS Agency
Review Committee; the second OMB/interagency review managed by OMB;
revisions in response to comments received from agency and OMB/
interagency review; negotiations among EPA, OMB, and the other federal
agencies over changes; and briefing of the Assistant Administrator for
ORD. It is concluded when ORD, the IRIS Agency Review Committee, and
other federal agencies resolve all issues, OMB informs EPA that issues
have been resolved, and a revised IRIS assessment is sent for
preparation of a final IRIS assessment.
[I] This phase encompasses the final preparation of the IRIS summary
and toxicological review and the review and clearance by the NCEA
Director. It is concluded when the assessment is posted on the IRIS Web
site after the Assistant Administrator, ORD, approves a fact sheet
about the assessment prepared by the chemical manager.
[J] EPA completed two assessments in July and September 2007.
[End of figure]
In addition to the current backlog of 70 ongoing and suspended IRIS
assessments, EPA data from 2001 through 2003 indicated that 287 of the
chemicals in the IRIS database may potentially need to be updated.
Specifically, EPA reviewed the scientific literature on the 460
chemicals in the database not being reassessed to identify assessments
that may need to be updated in light of new studies or information that
could potentially change the risk estimates currently in the IRIS
assessments. In addition, while conducting these literature reviews,
EPA identified new studies or information that would enable the agency
to develop additional risk estimates (e.g., add an estimate, such as an
RfD or cancer potency estimate, for assessments lacking such
estimates). EPA's "screening level review" found new information that
could potentially (1) change an existing risk estimate for 169
chemicals and/or (2) allow EPA to develop additional risk estimates for
210 chemicals. Although EPA identified these chemicals as candidates
for reassessment, as of fiscal year 2007, the agency had initiated
reassessments of only a few of these chemicals. Further, because the
screening levels were completed more than 4 years ago, it is likely
that more assessments may now be outdated.
EPA officials said they have initiated another screening level review
that they expect to be completed by July 2008. Importantly, in its
performance assessment of EPA's human health risk assessment program,
which includes IRIS, OMB considers health assessment values in IRIS as
out of date if they were developed more than 10 years ago and where new
scientific information has been identified that could change the health
assessment value. This designation highlights the need for EPA to
become productive in completing its IRIS assessments if it is to remain
a viable and credible assessment resource. We note that EPA recently
formed a new working group that will specifically address chemical
assessments that are over 10 years old and for which new information is
available. This group's efforts are currently in the planning stages,
and an EPA official involved in this group indicated that the agency
expects to begin work on this effort by March 2008.
Although EPA has stated its intent that the IRIS database be updated
and expanded to include new assessments requested by IRIS users as soon
as "practically possible," the current backlog continues to present a
practical impediment to doing so. Because of the backlog, for example,
EPA did not initiate new assessments in fiscal years 2006 and 2007.
[Footnote 7] In fact, key IRIS users--two EPA program offices--
specifically requested that the IRIS program management focus its
resources on expediting the completion of ongoing IRIS assessments
important to their regulatory and cleanup responsibilities rather than
initiating new assessments in fiscal year 2007. For example, the Office
of Solid Waste and Emergency Response identified 29 assessments that
were important to that office and asked EPA to finish these IRIS
assessments before initiating others. In addition, the Office of Air
identified 28 high-priority IRIS assessments that it needed to fulfill
its regulatory mandates,[Footnote 8] 14 of which it identified as being
of the "highest priority" and the other as "high priority;" IRIS
reviews for 12 of the chemicals of highest priority are in progress,
but most of these assessments are still either being drafted or are
undergoing an internal agency review. In terms of new assessments that
will be needed for EPA's air toxics program, an official in the Office
of Air and Radiation said there were about 12 to 20 air toxics of
concern nationwide for which IRIS risk information is needed and about
50 or so uncommon air toxics emitted in isolated "hot spots" (e.g.,
near a chemical plant in sparsely populated areas) where a small number
of people who live nearby--perhaps 200 or so--are exposed to relatively
high doses of these air toxics. He said it is more difficult to get
approval for assessments of such chemicals because of the small number
of people potentially affected, even though these chemicals may be very
harmful to human health and the environment.
Moreover, as discussed above, EPA program offices and state and local
entities have identified needs for assessments of hundreds of chemicals
not yet in IRIS. For example, in its 2003 needs assessment report, EPA
reported that about half of the chemicals and chemical classes that
IRIS users had nominated for assessment were not included in IRIS.
[Footnote 9] Reasons that IRIS users nominated these chemicals included
to support anticipated rule making, to support agency or state
implementation priorities, to address children's health concerns, and
to address the widespread occurrence of a chemical at contaminated
sites and in groundwater. Unmet IRIS needs affect IRIS data users both
within and outside EPA. For example, EPA's Office of Research and
Development noted in its 2003 Air Toxics Multi-Year Plan that
quantitative assessment values (for noncancer and cancer health
effects) for many high-priority air toxics that are needed to support
site-specific efforts and regulatory decisions are "missing." Along
these same lines, in its Office of Solid Waste Integrated Research and
Development Plan for the Hazardous Waste Identification Rule, EPA's
Office of Research and Development identified 460 chemicals of
potential concern and reported that roughly 200 of these lack
quantitative assessment values (estimates of the risk of cancer and
noncancer effects of the chemicals). Non-EPA users also are affected by
the lack of IRIS values. According to the preliminary results of an EPA
project aimed at determining how IRIS is used by non-EPA decision
makers,[Footnote 10] non-EPA users have one primary criticism regarding
IRIS--they are frustrated by the lack of new assessments, particularly
for "controversial" chemicals. These users reported that the absence of
an IRIS assessment creates enormous challenges for state regulatory
agencies and significant uncertainty for regulatory parties.
Although EPA's 2003 needs assessment report had identified a potential
need to complete 50 IRIS assessments annually to meet user needs, the
agency has not finalized more than 11 assessments a year during the
past 10 years. As figure 3 shows, productivity has, in fact, declined
since 2003, with EPA completing 4 assessments in fiscal year 2005 and 2
each in fiscal years 2006 and 2007. EPA's updated multiyear plan
estimates incremental increases in the number of IRIS assessments EPA
will complete in fiscal years 2008 through 2011, at which point EPA's
annual goal is to complete 16 IRIS assessments.[Footnote 11] We note
that even if EPA were to overcome the significant productivity
difficulties it has experienced in recent years and meet this goal of
completing 16 assessments a year beginning in 2011, it is not clear
that this level of productivity would meet the needs of EPA program
offices and other users, given the current status of the database. For
example, in November 2007, the Deputy Administrator of EPA's Region 2
told the Board of Scientific Counselors panel reviewing EPA's human
health risk assessment program that even 16 IRIS assessments per year
would not meet their chemical assessment needs. Although funding for
the IRIS program has increased since 2000, we note that fiscal year
2007 funding of $9.6 million represents approximately 0.1 percent of
EPA's $7.3 billion annual budget.
It is a positive step that EPA delivered 32 IRIS assessments to OMB for
OMB/interagency review in fiscal years 2006 and 2007. However, in
general, the IRIS assessments sent to OMB during this period were on
less controversial chemicals. In its December 2005 multiyear plan, EPA
specified 10 major assessments that would be sent for external review
in fiscal years 2006 and 2007. As of December 1, 2007, only 2 of these
assessments had been sent for OMB/interagency review.[Footnote 12] In
order to meet user needs and therefore enable EPA to more effectively
protect public health, it will be important for EPA to make progress on
the assessments of the more controversial chemicals, which tend to be
those to which people are more widely exposed.
Lastly, while EPA's assessment process changes have resulted in EPA
providing the public with more information about IRIS assessments, some
of the information provided in Federal Register notices and in the IRIS
database itself is not accurate or is incomplete. For example, some
assessments that have been suspended have continued to be identified as
ongoing, and information on the status of the individual assessments
provided in a system called IRIS Track has not been kept up to date.
Regarding this latter point, we found that more EPA chemical managers
started updating this database as a result of our review. Nonetheless,
some of the milestone estimates remained outdated, and the milestones
in IRIS Track did not reflect the current assessment process. We note
that in November 2007, EPA created a new IRIS Web page, updating and
consolidating some of the milestones to better reflect the current
process. However, two key milestones--the second OMB/interagency and
EPA internal agency reviews--are still not identified.
EPA's Productivity Problems Stem from Several Key Factors:
In our view, several key factors have contributed to EPA's inability to
achieve a level of productivity that is needed to sustain the IRIS
program and database. These factors that have hindered EPA's efforts to
improve productivity are:
* the OMB/interagency review process managed by OMB,
* the growing complexity and scope of risk assessments,
* certain management decisions and issues regarding the IRIS program,
* congressional action that has delayed some assessments with
potentially significant economic effects, and:
* the compounding effect of delays.
OMB/Interagency Review Process:
One factor that has made it more difficult for EPA to complete IRIS
assessments in a timely manner is the OMB-managed interagency review
process, initiated in 2004 at OMB's direction. According to OMB, the
purpose of OMB/interagency reviews is to ensure that federal agencies
are aware of draft IRIS assessments in which they have an interest and
that these agencies have the opportunity to be involved with the IRIS
assessments as they are being developed in order to help ensure and
increase their quality. This process, initially conducted on an ad hoc
basis, was put in place in response to interagency conflicts that EPA
faced when it attempted to finalize some IRIS assessments for chemicals
that became highly controversial, such as perchlorate, naphthalene, and
TCE--chemicals that are or have been considered by some federal
agencies, including DOD, DOE, and NASA, to be integral to their
missions. Notably, EPA's IRIS assessments of these chemicals could lead
to regulatory actions that could, among other things, restrict the use
of these chemicals, require agencies to provide protective gear to
their employees exposed to the chemicals at work, or require agencies
or their contractors to carry out or pay for cleanup of contamination
at federal sites. The interagency conflicts about these IRIS
assessments have contributed to their delays--resulting, for example,
in EPA having to essentially restart the naphthalene assessment after
it had been drafted and peer reviewed.
The OMB/interagency review process has evolved over time, but it
remains an informal process with OMB generally communicating its review
requirements verbally. OMB has not provided EPA with written guidance,
directives, policies, or procedures on this significant review program.
In 2005, OMB started requiring EPA to send OMB all draft IRIS
assessments for an OMB/interagency review before the assessments are
provided to external peer review panels. In addition, OMB requires EPA
to send the assessments to OMB a second time, after EPA has addressed
the external peer review comments and recommendations. According to EPA
officials, in 2007, OMB informed EPA that it cannot send its draft
assessments for external peer review or post final EPA assessments on
the EPA IRIS database until OMB verbally informs EPA that it has
satisfactorily addressed OMB/interagency comments. OMB has not
specified which authority or authorities it is using to review IRIS
assessments. Because IRIS assessments are not regulations, they are not
covered by Executive Order 12866 which provides for OMB review of
proposed regulations, among other things. In addition, although in
January 2007, Executive Order 13422 amended Executive Order 12866 by
establishing a role for OMB in reviewing "significant guidance"
documents, which could potentially include IRIS assessments, OMB
officials told us that OMB has not formally classified IRIS assessments
as significant guidance documents within the meaning of this executive
order.[Footnote 13] These officials said that OMB had the authority to
review IRIS assessments prior to the issuance of Executive Order 13422
and was continuing to use this general authority. We note that these
executive orders address only OMB's reviews of two specific categories
of agency documents and that OMB has not identified IRIS assessments as
falling into either of these categories.
Under the OMB/interagency review process for IRIS assessments, OMB
identifies the federal agencies from which it will seek comments on the
assessments and provides comments and questions to EPA from OMB and the
other federal agencies. EPA then revises the assessments to address the
comments and questions and also provides OMB with a document discussing
how it has addressed each of the issues raised. According to EPA
officials, this step is concluded when OMB verbally informs EPA that
all of the issues are resolved. Thus, before EPA can provide its draft
assessments to external peer reviewers, EPA, OMB, and the other federal
agencies must reach agreement on (1) EPA's revised IRIS assessment and
(2) the scope of the external peer review, including specific questions
reviewers will be asked.
The OMB reviews have primarily been conducted by a policy analyst in
the Office of Information and Regulatory Affairs who, according to OMB
officials, has a toxicology background and is qualified to conduct
these reviews. If EPA does not agree with some of the proposed changes
or peer review requirements, OMB officials said the assessment could be
elevated to higher management levels within EPA for resolution. Thus,
the framework of the OMB/interagency review process can essentially
give more weight to OMB and the other federal agencies than to EPA
chemical managers who have prepared the assessments and the many
scientists and experts who have peer reviewed them. Further, in
commenting on OMB's proposed risk assessment bulletin in 2007, the
National Academies stated its concern that scientific issues may be
superseded by policy considerations to the extent that the technical
aspects of risk assessments would be overseen by OMB and not by the
peer review process or by agency technical managers. Similarly, an EPA
official noted that the farther removed the scientists and experts who
have prepared or peer reviewed the assessments are from the
negotiations and decisions over assessment changes requested by OMB and
other federal agencies, the decisions are based more on political
rather than scientific considerations.
According to EPA officials, some of the OMB/interagency reviews have
provided valuable input. However, they said the reviews have also added
a significant amount of time to the assessment process, in part because
the reviews are not subject to any specific time frames or deadlines
and because responding to OMB comments in some cases has required
several iterations to address nonsubstantive issues involving minor
clarifications, semantics, and organization. (We note that according to
the executive order under which OMB reviews regulatory actions, OMB
must generally complete such reviews within 90 days.) In terms of time
frames, our analysis of EPA data as of December 1, 2007, indicate that
of the 36 assessments sent to OMB for the first OMB/interagency review,
16 have completed this process. While 4 completed the process in less
than 6 months, 12 of them were in the review process for 6 or more
months. Moreover, five additional assessments--the previously discussed
assessments of acute exposure--were discontinued at the direction of
OMB after a year in the OMB/interagency review stage.
In addition, as of December 1, 2007, 5 of 10 draft assessments had
completed the second OMB/interagency review, which OMB officials said
is conducted by OMB to ensure that EPA has adequately considered the
comments from external peer review panels. These assessments were at
the second OMB/interagency review stage for periods ranging from 10
days to almost 4 months. According to EPA, while agencies should only
point out major scientific issues that would warrant halting the
release of the assessment, some comments submitted to them from the
second OMB/interagency reviews by OMB have gone beyond their intended
purpose of identifying only major scientific concerns and thus have
unnecessarily added time to the assessment process. (See app. III for
additional information about the time frames for the OMB/interagency
reviews.) Further, EPA officials noted that the OMB/interagency review
process can delay not only the IRIS assessments undergoing OMB/
interagency review but also the other assessments that the EPA chemical
managers would be working on or starting if they were not engaged in
responding to the OMB/interagency comments.
Overall, the two OMB/interagency reviews managed by OMB have introduced
a significant level of uncertainty into the time frames for completing
IRIS assessments. This fact was reflected in EPA's December 2005 Human
Health Risk Assessment Multi-Year Plan, which identified EPA's annual
performance goals for fiscal years 2006 through 2012. The goals did not
include the number of IRIS assessments completed and posted on the IRIS
Web page--the performance goal that would be expected--but instead the
number of assessments provided to OMB for the first OMB/interagency
review. While EPA's annual performance goal for IRIS assessments had
been the number of completed assessments, during the Program Assessment
Rating Tool (PART) review by OMB, it was agreed that the number of
assessments provided to OMB for OMB/interagency review was the most
appropriate annual measure of performance because EPA "relinquishes
direct control of production dates" when it sends draft IRIS
assessments to OMB.
In addition to adding time to the IRIS assessment process, the OMB/
interagency review process also affects the credibility of assessments
primarily because the review process lacks transparency. According to
EPA's Risk Characterization Policy and Handbook, the risk assessment
process--which includes the hazard identification and dose-response
analysis that constitutes IRIS assessments--should be transparent and
its products should be clear, consistent, and reasonable. Transparency
is particularly important in cases such as this when potentially
affected parties are providing input into, and in some cases
questioning, EPA's scientific analyses supporting its IRIS assessments.
Specifically, a transparent process allowing IRIS users and the public
to see the comments from OMB and other federal agencies--including
those potentially impacted by the IRIS assessments--as well as EPA's
responses to them, could help alleviate concerns about potential bias
in the assessments.
However, under the OMB/interagency review process managed by OMB, the
comments EPA receives from OMB and other federal agencies and EPA's
responses to them are not available to the public. OMB does not
authorize their disclosure to the public on the basis that these
communications are internal deliberations of the executive branch.
Overall, because the rationale for changes to EPA's scientific
assessments stemming from OMB and the interagency review process are
not disclosed, the credibility of the IRIS assessments is reduced. We
note that the former Assistant Administrator for the Office of Research
and Development has emphasized the importance of transparency in the
IRIS assessment process. Specifically, he stated that the best cure for
controversy surrounding IRIS assessments "is early and frequent visits
to the experts, second opinions, and lots of sunshine" (that is,
transparency).
Growing Complexity of Risk Assessments and Risk Assessment Methods and
Models:
Another factor that affects the length of time it takes to complete
IRIS assessments is the growing scientific complexity of the
assessments. For example, according to John Graham, the former director
of both Harvard's Center for Risk Analysis and OMB's Office of
Information and Regulatory Affairs, more and different types of
scientific data have made the IRIS assessment of health effects more
challenging.[Footnote 14] As a result, EPA chemical managers and
scientists need to explore new methods of analyses and evaluate a wider
variety of potential health effects (e.g., multiple disease endpoints)
than in the past. In addition, chemical managers responsible for the
assessments are working with a growing body of complex risk assessment
guidelines, such as EPA's 2005 Guidelines for Carcinogen Risk
Assessment (final revised cancer assessment guidelines) and its
supplement relating to children, Supplemental Guidance for Assessing
Susceptibility from Early Life to Exposure to Carcinogens. In addition,
chemical managers must increasingly analyze studies using state-of-the-
art physiologically based pharmacokinetic (PBPK) models, which improve
the estimation of doses across species and routes of exposure and
provide insights on uncertainty.[Footnote 15] Before EPA relies on PBPK
models intended for risk assessments, the agency needs to evaluate
them. Such evaluations include a review of the model purpose,
structure, mathematical representation, parameter estimation
(calibration), and computer implementation. EPA has established
criteria for acceptance of a PBPK model for risk assessment purposes in
one of two detailed reports issued in 2006, Approaches for the
Application of Physiologically Based Pharmacokinetic (PBPK) Models and
Supporting Data in Risk Assessment. The other 2006 report on PBPK
models is the Use of Physiologically Based Pharmacokinetic (PBPK)
Models to Quantify the Impact of Human Age and Interindividual
Differences in Physiology and Biochemistry Pertinent to Risk. In fiscal
years 2008 and 2010, EPA plans to issue additional information on PBPK
modeling for use in risk assessments.
The chemical managers are also called upon to use some new methods and
models of risk assessment that are being developed or being applied for
the first time in the absence of guidelines. For example, uncertainty
analysis is a method in the early stages of development and use in
risks assessments, for which EPA has not yet developed guidance.
[Footnote 16] However, chemical managers are having to develop and
apply new approaches to quantify and communicate uncertainty. According
to EPA, these approaches include identifying alternative studies and
endpoints for the application of uncertainty factors in noncancer risk
assessment and the application of alternative dose- response models for
cancer risk assessment. Important cutting-edge assessment models
relevant to IRIS assessments include biologically based dose-response
models, which are based on the modes of action of chemicals--that is,
analysis of physiological, chemical, and biological information that
helps identify a chemical agent's role in the development of tumors.
These highly complex models can analyze multiple modes of action and
can be responsive to EPA's 2005 guidelines for cancer assessment that
emphasize mode-of-action evaluation. According to EPA, the agency has
extensive experience in qualitative mode-of- action analysis but more
limited experience in the quantitative evaluation of multiple modes of
action.
EPA Management Decisions and Issues:
Some of the IRIS assessment process changes that EPA management has
implemented in recent years have made it more difficult for the agency
to complete assessments in a timely manner. These changes were aimed
primarily at improving the quality of the assessments. It is too early
to determine whether or to what extent the changes have enhanced the
scientific credibility of the assessments,[Footnote 17] but it is clear
they have contributed to assessment delays overall. Changes that have
affected the time frames for IRIS assessments include waiting for
additional scientific studies to be completed, waiting for the
development of an acceptable methodology for presenting qualitative and
quantitative uncertainty analysis in the IRIS assessments, and numerous
process changes.
Waiting for additional scientific studies: EPA management's decision in
some cases to suspend ongoing IRIS assessments while waiting for
additional scientific studies to be completed has contributed to EPA's
lack of productivity. According to a former IRIS program director,
EPA's general approach in the 1990s was to use only information from
completed scientific studies available at the time of the assessment--
e.g., the reviews were based on the best available science. However,
EPA has awaited the results of new and ongoing studies before
completing some IRIS assessments, which has resulted in delaying them
for years. Examples of key chemical assessments that have been delayed
while EPA waits for new studies include those for formaldehyde and
Royal Demolition Explosive,[Footnote 18] discussed later. Other delayed
assessments include those for tetrahydrofuran, perfluorooctane
sulfonate-potassium salt (PFOS), and perfluorooctanoic acid-ammonium
salt (PFOA). We understand that there may be exceptional circumstances
for which it may be appropriate to wait for the results of an important
ongoing study, such as a major epidemiological study that will provide
new, critical data for an assessment. According to EPA officials, this
is the case with research they are awaiting for its IRIS assessment of
asbestos. However, as a general rule, requiring that IRIS assessments
be based on the best science available at the time of the assessment,
as had been the prior practice, is a standard that would best support a
goal of completing assessments within reasonable time periods and
minimizing the need to conduct significant levels of rework.
Developing an acceptable uncertainty analysis for IRIS assessments:
Another management decision that has delayed the completion of IRIS
assessments is to incorporate comprehensive uncertainty analysis into
the IRIS assessments. Peer reviews of EPA's assessments by the National
Academies and others have sometimes recommended additional uncertainty
analysis; the Assistant Administrator, Office of Research and
Development, has made the inclusion of qualitative and quantitative
uncertainty analyses in IRIS assessments a high priority to, among
other things, support better decisions and guide EPA's research agenda.
Such analyses require the use of state-of-the-art tools to quantify
uncertainty. That is, comprehensive--and, in particular, quantified--
uncertainty analysis, is an emerging analytic tool. For example, EPA's
August 2007 Human Health Risk Assessment Multi-Year Plan estimates
releasing an external review draft report on methods for analyzing and
characterizing uncertainty in hazard and dose-response assessments in
2010; the plan--which covers 2006 through 2012--does not estimate a
final report date for this important guidance on uncertainty analysis.
In the interim, EPA chemical managers have had to try to develop
complex uncertainty estimates in draft assessments in the absence of
agency guidance or protocols. As discussed later, this requirement has
delayed the completion of an important assessment (tetrachloroethylene)
for almost two years. Moreover, because EPA is now requiring all IRIS
assessments to include basic or enhanced uncertainty analysis, other
significant assessments that have been drafted have also been delayed,
pending approval by the Assistant Administrator, Office of Research and
Development, of an acceptable template for uncertainty analysis for
significant assessments. Further, external peer reviewers of IRIS
assessments containing quantified uncertainty analyses will need to
have specialized expertise to assess the quality and reliability of
these cutting-edge analyses--which themselves contain uncertainties and
incorporate numerous assumptions. Evaluating the uncertainty
assessments will be challenging, given their complexities and the lack
of guidance on this emerging method. Along these lines, members of the
Board of Scientific Counselors[Footnote 19] have questioned whether
expert peer reviewers of EPA's IRIS assessments will have the necessary
expertise in this emerging method. Finally, effectively communicating
the results of complex uncertainty analyses may be challenging--in
fact, some have questioned whether highly detailed quantitative
uncertainty analysis enhances the values of risk assessments. For
example, in commenting on a proposed OMB bulletin that would have
provided risk assessment guidance to federal agencies, the Department
of Health and Human Services noted that "characterization of every
possible uncertainty or extensive evaluation of each assumption . . .
could result in a confusing, less straight-forward document . . . that
would not serve the public or the risk assessment community well."
[Footnote 20] Similarly, in its review of the proposed bulletin, the
National Academies concluded that "there is a serious danger that
agencies will produce ranges of meaningless and confusing risk
estimates, which could result in risk assessments of reduced rather
than enhanced quality and objectivity."
Continuous process changes, outdated standard operating procedures, and
management issues: EPA's continual changes to the IRIS assessment
process present a challenge to the chemical managers who are
undertaking the assessments. Further, a number of changes have been
implemented informally since the last update to the agency's standard
operating procedures for fiscal year 2006 reviews. According to EPA,
these changes have been made in order to continually improve the
assessment process and to respond to changing requirements, such as the
OMB-managed OMB/interagency review and the need to incorporate emerging
risk assessment methods. Chemical managers have told us that there is
much confusion about the process, including who needs to be briefed on
the draft assessments and who has approval authority to move an
assessment to the next step. In fact, in the absence of current
operating procedures to guide chemical managers on basic procedures and
program management responsibilities for the development, review, and
finalization of IRIS assessments, EPA chemical managers recently took
the initiative to develop a detailed graphic providing their best
understanding of the process. One aspect of the IRIS assessment process
that has become particularly unclear is the steps required to obtain
approvals to move draft assessments through various stages (e.g., to
OMB, external peer review, or final issuance). For example, until 2004,
EPA used an agencywide consensus process to reach either unanimous or
substantial agreement on draft assessments among members of the IRIS
Agency Review Committee; if agreement could not be reached by the
committee, the operating procedures had provided a detailed process for
resolving contentious issues. When EPA moved from the consensus
approach to an "agency resolution" approach, however, the guidance said
only that final decisions on any contentious issues were to be resolved
by either the NCEA Director or the Agency Science Advisor.[Footnote 21]
In addition, while approval to move draft assessments to external peer
review had been made by the NCEA Director, according to EPA officials,
in 2006, the Assistant Administrator started approving drafts for the
first of three external reviews of IRIS assessments--the initial OMB/
interagency review. However, this change occurred informally and was
not reflected in the existing standard operating procedures.[Footnote
22] We also note that, starting in 2004, chemical managers were
required to provide a fact sheet on final IRIS assessments to the
Assistant Administrator for review and approval before the assessments
could be posted on the Web site.[Footnote 23] Raising the level at
which various approvals are made has added time to the process because
it involves additional briefings, additional revisions and negotiations
over revisions, and, at times, delays due to scheduling difficulties.
As a result, it is important for EPA to both articulate and evaluate
its internal review process to help ensure the appropriate balance
between product quality and timeliness.
Finally, in addition to process and approval clarifications, it also
appears that the IRIS program could benefit from more rigorous
management attention and oversight. For example, the risk assessment
for chloroform--identified as one of the highest-priority assessments
requested by the EPA Office of Air and Radiation--was first announced
in a January 1998 Federal Register notice; however, the chemical
manager for the chloroform assessment retired from the agency in late
2004, and a new chemical manager was not assigned to this assessment
until early 2007.[Footnote 24] Although EPA continued to identify this
assessment as ongoing, it was actually unstaffed and therefore
suspended for at least 2 years. In addition, a relatively simple, less
controversial assessment of 2,2,4-trimethylpentane, which concluded
that there were insufficient scientific data to either quantify the
noncancer health risks associated with the chemical or to assess its
potential to cause cancer, took more than 4 years to complete. EPA
officials could not explain why this assessment--which should have
taken less than a year to complete--took so long, aside from noting
that the responsible staff had been given other high-priority
assignments during this time. EPA needs to ensure that its IRIS
assessments are given high priority and adequately staffed so that
costly delays are avoided.
Congressional Actions:
Another factor that can delay the completion of IRIS assessments is the
potential for congressional involvement in assessments that become
controversial, such as those with potentially significant economic
effects. Because of the potential for such assessments to lead to
regulatory actions that can significantly affect certain industries or
federal agencies, it is particularly important that these assessments
effectively and appropriately use the best available science. Even
assuming the best available science, however, uncertainties remain an
inherent aspect of complex chemical risk assessments. As the National
Academies has noted, "A risk assessment usually involves incomplete
data, scientific uncertainty, and the need for expert judgment" and
"almost every risk assessment is open to challenge on one ground or
another."[Footnote 25] As a result, the assessments may be questioned
on various scientific and technical grounds and subjected to intense
national scrutiny by many individuals and organizations, including the
media. Constituents may contact their elected representatives with
their concerns, and politicians are likely to become involved, either
supporting the content of the assessments or challenging it. Because of
this debate, Members of the Congress, congressional committees, or the
Congress as a whole may direct EPA to take certain steps before
finalizing a particular assessment. While the intense scrutiny to which
some chemical assessments are subjected can result in improved
assessments, all uncertainties cannot be eliminated and controversies
can continue beyond the point at which additional analysis is helpful.
Further, addressing questions and concerns about assessments often
involves considerable rework and takes a significant amount of time.
EPA strives to balance the desire for the best possible assessment with
its responsibilities for protecting the public health, which it can
only do with timely assessments of chemicals. That being said, it can
be challenging for EPA to both identify and achieve the proper balance
between these competing goals.
In the case of certain controversial chemical assessments, actions by
congressional committees and individual members have led EPA to, for
example, postpone completion of the IRIS assessment of formaldehyde for
years until an update of an epidemiological study that had just been
released was completed. Another response to congressional concerns is
EPA's decision to reconsider the quantitative noncancer assessment of a
chemical, dibutyl phthalate, that had completed all internal and
external reviews and was ready to be released in 2007. The noncancer
assessment, an update to the assessment completed in 1990, would have
allowed more exposure to dibutyl phthalate than the earlier assessment.
In 2000, the Department of Health and Human Services' National
Toxicology Program concluded that dibutyl phthalate may adversely
affect human reproduction or development if exposures are sufficiently
high. This chemical, regulated under the Clean Air Act's air toxics
program, is used in many consumer products, such as cosmetics, nail
polishes, fragrances, wood stains, and toys; the European Union has
banned its use in cosmetics and restricted its use in children's toys
containing concentrations of phthalates in excess of 0.1 percent.
Similarly, after January 1, 2009, a California statute enacted in 2007
will prohibit the manufacture, sale, or distribution of certain toys
and child care articles if the products contain concentrations of
phthalates exceeding 0.1 percent.[Footnote 26] Because of congressional
questions and comments from the Natural Resources Defense Council
about, among other things, the adequacy of the uncertainty factors used
in the analysis and the lack of consideration in the draft assessment
of cumulative exposure to multiple phthalates (similar chemicals in the
same class) that have similar health effects, EPA has suspended the
assessment. In December 2007, an EPA official said that EPA has decided
to seek advice from the National Academies on whether the agency should
develop a new type of assessment for this chemical--one that would
assess a class of related chemicals. If EPA decides to develop a new,
cumulative IRIS assessment, it will likely be many years before an
updated IRIS assessment addressing dibutyl phthalate is completed.
[Footnote 27]
Compounding Effect of Delays:
An overarching factor that affects EPA's ability to complete IRIS
assessments in a timely manner is that once a delay in the assessment
process occurs, work that has been completed can become outdated,
necessitating rework throughout some or all of the assessment process.
For example, delays often require repeating reviews of the scientific
literature on a chemical to take into account the time that has passed
since the literature review was completed; this, in turn, may require
detailed analyses of any new studies found to be relevant. Moreover,
new risk assessment guidelines and cutting-edge methodologies that the
agency has started applying to other assessments may now need to be
applied to an assessment being reworked to meet current assessment
requirements and standards and the expectations of peer reviewers. Once
these analyses are complete, the draft will likely need to be revised
and again subjected to internal reviews, OMB/interagency reviews, and
scientific peer review. In addition, the longer an assessment is in
progress, the more likely it becomes that staff will change due to
retirements and resignations; in turn, newly assigned chemical managers
face a learning curve in reviewing often voluminous data and analyses.
For example, at least four chemical managers have been assigned to the
nitrobenzene assessment since it was started in 1998. Overall, even a
single delay can have a domino effect--having far-reaching, time-
consuming consequences, in some cases, requiring that the assessment
process essentially start over. In addition, because chemical managers
must continue to devote time and attention to assessments that
experience delays--often repeating prior steps to update the
assessments--their ability to work on other ongoing assessments and
undertake new ones is limited.
Key IRIS Assessments Have Been Delayed by Some of These Factors:
Some key IRIS assessments have been in progress for a number of years,
in part because of delays stemming from one or more of the factors
discussed above. Examples include the following:
Naphthalene. EPA started the IRIS assessment of cancer risks stemming
from the inhalation of naphthalene in 2002. Naphthalene is used in jet
fuel and in the production of widely used commercial products such as
moth balls, dyes, insecticides, and plasticizers. According to a
presentation delivered at the 2007 annual meeting of the Society for
Risk Analysis by an Army Corps of Engineers toxicologist,[Footnote 28]
"The changing naphthalene regulatory environment includes a draft EPA
risk assessment that if/when finalized, will change naphthalene's
status from 'possible' to 'likely' human carcinogen."[Footnote 29]
Thus, according to this presentation, one potential impact of this IRIS
assessment on DOD is that DOD would need to provide many employees
exposed to naphthalene with equipment measuring their exposure to the
chemical. In addition, because many military bases are contaminated
with naphthalene, a component of jet fuel (approximately 1 percent to 3
percent) used by all DOD services, DOD could face extensive cleanup
costs. By 2004, 2 years after starting the assessment, EPA had drafted
a chemical assessment that had completed internal peer reviews and was
about to be sent to an external peer review committee. Once it returned
from external review, the next step, at that time, would have been a
formal review by EPA's IRIS Agency Review Committee. If approved, the
assessment would have been completed and released. However, in part
because of concerns raised by DOD, OMB asked to review the assessment
and conducted an interagency review of the draft. In their 2004 reviews
of the draft IRIS assessment, both OMB and DOD raised a number of
concerns about the assessment and suggested to EPA that it be suspended
until additional research could be completed to address what they
considered to be significant uncertainties associated with the
assessment. Although all of the issues raised by OMB and DOD were not
resolved, EPA continued with its assessment by submitting the draft for
external peer review, which was completed in September 2004.[Footnote
30] However, according to EPA, OMB continued to object to the draft
IRIS assessment and directed EPA to convene an additional expert review
panel on genotoxicity to obtain recommendations about short-term tests
that OMB thought could be done quickly.[Footnote 31] According to EPA,
this added 6 months to the process, and the panel, which met in April
2005, concluded that the research that OMB was proposing could not be
conducted in the short term. Nonetheless, EPA officials said that the
second expert panel review did not eliminate OMB's concerns regarding
the assessment, which they described as reaching a stalemate. In
September 2006, EPA decided, however, to proceed with developing the
assessment. By this time, the naphthalene assessment had been in
progress for over 4 years; EPA decided that the IRIS noncancer
assessment, issued in 1998, was outdated and needed to be revisited.
Thus, EPA expanded the IRIS naphthalene assessment to include both
noncancer and cancer assessments. As a result, 6 years after the
naphthalene assessment began, it is now back at the drafting stage. The
assessment now will need to reflect relevant research completed since
the draft underwent initial external peer review in 2004, and it will
have to undergo all of the IRIS assessment steps again, including
additional internal and external reviews that are now required. This
series of delays has limited EPA's ability to conduct its mission. For
example, the Office of Air and Radiation has identified the naphthalene
assessment as one of its highest-priority needs for its air toxics
program. In addition, the Office of Solid Waste and Emergency Response
considers the naphthalene assessment a high priority for the Superfund
program--naphthalene has been found in at least 654 of Superfund's
current or former National Priorities List sites.[Footnote 32] Although
EPA currently estimates that it will complete the assessment in 2009,
meeting this revised estimate will be challenging, given all of the
steps that are yet to be completed and the extensive external scrutiny
to which it will continue to be subjected.
Royal Demolition Explosive. This chemical, also called RDX or hexahydro-
1,3,5-trinitrotriazine, is a highly powerful explosive used by the U.S.
military in thousands of munitions. Currently classified by EPA as a
possible human carcinogen, this chemical is known to leach from soil to
groundwater. Royal Demolition Explosive can cause seizures in humans
and animals when large amounts are inhaled or ingested, but the effects
of long-term, low-level exposure on the nervous system are unknown. As
is the case with naphthalene, the IRIS assessment could potentially
require DOD to undertake a number of actions, including steps to
protect its employees from the effects of this chemical and to clean up
many contaminated sites. Although EPA started an IRIS assessment of
Royal Demolition Explosive in 2000, it has made minimal progress on the
assessment because EPA agreed to a request by DOD to wait for the
results of DOD-sponsored research on this chemical. In 2007, EPA began
to actively work on this assessment, although some of the DOD-sponsored
research is still outstanding.
Formaldehyde. EPA began an IRIS assessment of formaldehyde in 1997
because the existing assessment was determined to be outdated.[Footnote
33] Formaldehyde is a colorless, flammable, strong-smelling gas used to
manufacture building materials, such as pressed wood products, and used
in many household products, including paper, pharmaceuticals, and
leather goods. While EPA currently classifies formaldehyde as a
probable human carcinogen, the International Agency for Research on
Cancer (IARC), part of the World Health Organization, classifies
formaldehyde as a known human carcinogen. Since 1986, studies of
industrial of workers have suggested that formaldehyde exposure is
associated with nasopharyngeal cancer, and possibly with leukemia. For
example, in 2003 and 2004, the National Cancer Institute (NCI) and the
National Institute of Occupational Safety and Health (NIOSH) released
epidemiological studies following up on earlier studies tracking about
26,000 and 11,000 industrial workers, respectively, exposed to
formaldehyde; the updates showed exposure to formaldehyde might also
cause leukemia in humans, in addition to the cancer types previously
identified. According to NCI officials, the key findings in their
follow-up study were an increase in leukemia deaths and, more
significantly, an exposure/response relationship between formaldehyde
and leukemia--as exposure increased, the incidence of leukemia also
rose. As with the earlier study, NCI found more cases of a rare form of
cancer, nasopharyngeal cancer, than would usually be expected. The
studies from NCI and NIOSH were published in 2003 and 2004,[Footnote
34] around the time that EPA was still drafting its IRIS assessment. In
November 2004, the Chairman of the Senate Environment and Public Works
Committee requested that EPA delay completion of its IRIS assessment
until an update to the just-released NCI study could be conducted,
indicating that the effort would take, at most, 18 months. EPA agreed
to wait--and more than 3 years later, the NCI update is not yet
complete. As of December 2007, NCI estimates that the study will be
completed in two stages, one in mid-2008 and the second one later that
year. An NCI official said that the additional leukemia deaths
identified in the update provide "greater power" to detect associations
between exposure to formaldehyde and cancer. EPA's inability to
complete the IRIS assessment it started more than 10 years ago in a
timely manner has had a significant impact on EPA's air toxics program.
Specifically, when EPA promulgated a national emissions standard for
hazardous air pollutants covering facilities in the plywood and
composite wood industries in 2004, EPA's Office of Air and Radiation
took the unusual step of not using the existing IRIS estimate but
rather decided to use a cancer risk estimate developed by an industry-
funded organization, the CIIT Centers for Health Research (formerly,
the Chemical Industry Institute of Toxicology) that had been used by
the Canadian health protection agency. The IRIS cancer risk factor had
been subject to criticism because it was last revised in 1991 and was
based on data from the 1980s. In its final rule, EPA stated that "the
dose-response value in IRIS is based on a 1987 study, and no longer
represents the best available science in the peer-reviewed literature."
The CIIT quantitative cancer risk estimate that EPA used in its health
risk assessment in the plywood and composite wood national emissions
standard indicates a potency about 2,400 times lower than the estimate
in IRIS that was being re-evaluated and that did not yet consider the
2003 and 2004 NCI and NIOSH epidemiological studies. According to an
EPA official, an IRIS cancer risk factor based on the 2003 and 2004 NCI
and NIOSH studies would likely be close to the current IRIS assessment,
which EPA has been attempting to update since 1997. The decision to use
the CIIT assessment in the plywood national emissions standard was
controversial, and officials in EPA's National Center for Environmental
Assessment said the center identified numerous problems with the CIIT
estimate. Nonetheless, the Office of Air and Radiation used the CIIT
value, and that decision was a factor in EPA exempting certain
facilities with formaldehyde emissions from the national emissions
standard. In June 2007, a federal appellate court struck down the rule,
holding that EPA's decision to exempt certain facilities that EPA
asserted presented a low health risk exceeded the agency's authority
under the Clean Air Act.[Footnote 35] Further, the continued delays of
the IRIS assessment of formaldehyde--currently estimated to be
completed in 2010 but after almost 11 years still in the draft
development stage--will impact the quality of other EPA regulatory
actions, including other air toxics rules and requirements.
Trichloroethylene. Also known as TCE, this chemical is a solvent widely
used as a degreasing agent in industrial and manufacturing settings; it
is a common environmental contaminant in air, soil, surface water, and
groundwater. TCE has been linked to cancer, including childhood cancer,
and other significant health hazards, such as birth defects. TCE is the
most frequently reported organic contaminant in groundwater, and
contaminated drinking water has been found at Camp Lejeune, a large
Marine Corps base in North Carolina. TCE has also been found at
Superfund sites and at many industrial and government facilities,
including aircraft and spacecraft manufacturing operations. In 1995,
the International Agency for Research on Cancer classified TCE as a
probable human carcinogen, and in 2000, the Department of Health and
Human Services' National Toxicology Program concluded that it is
reasonably anticipated to be a human carcinogen. Because of questions
raised by peer reviewers about the IRIS cancer assessment for TCE, EPA
withdrew it from IRIS in 1989 but did not initiate a new TCE cancer
assessment until 1998. In 2001, EPA issued a draft IRIS assessment for
TCE that proposed a range of toxicity values indicating a higher
potency than in the prior IRIS values and characterizing TCE as "highly
likely to produce cancer in humans." The draft assessment, which became
controversial, was peer reviewed by EPA's Scientific Advisory Board and
released for public comment. A number of scientific issues were raised
during the course of these reviews, including how EPA had applied
emerging risk assessment methods--such as assessing cumulative effects
(of TCE and its metabolites) and using a physiologically based
pharmacokinetic model--and the uncertainty associated with the new
methods themselves.[Footnote 36] To help address these issues, EPA,
DOD, DOE, and NASA sponsored a National Academies review to provide
guidance. The National Academies report, which was issued in 2006,
concluded that the weight of evidence of cancer and other health risks
from TCE exposure had strengthened since 2001 and recommended that the
risk assessment be finalized with currently available data so that risk
management decisions could be made expeditiously. The report
specifically noted that while some additional information would allow
for more precise estimates of risk, this information was not necessary
for developing a credible risk assessment. Nonetheless, 10 years after
EPA started its IRIS assessment, the TCE assessment is back at the
draft development stage. EPA estimates this assessment will be
finalized in 2010. More in line with the National Academies'
recommendation to act expeditiously, five senators introduced a bill in
August 2007 that, among other things, would require EPA to both
establish IRIS values for TCE and issue final drinking water standards
for this contaminant within 18 months.
Tetrachloroethylene. EPA started an IRIS assessment of
tetrachloroethylene--also called perchloroethylene or "perc"--in 1998.
Tetrachloroethylene is a manufactured chemical widely used for dry
cleaning of fabrics, metal degreasing, and making some consumer
products and other chemicals. Tetrachloroethylene is a widespread
groundwater contaminant, and the Department of Health and Human
Services' National Toxicology Program has determined that it is
reasonably anticipated to be a carcinogen. The IRIS database currently
contains a 1988 noncancer assessment based on oral exposure that will
be updated in the ongoing assessment. Importantly, the ongoing
assessment will also provide a noncancer inhalation risk and a cancer
assessment. The IRIS agency review of the draft assessment was
completed in February 2005, the draft assessment was sent to OMB for
OMB/interagency review in September 2005, and the OMB/interagency
review was completed in March 2006. EPA had determined to have the next
step, external peer review, conducted by the National Academies--the
peer review choice reserved for chemical assessments that are
particularly significant or controversial. EPA contracted with the
National Academies for a review by an expert panel, and the review was
scheduled to start in June 2006 and be completed in 15 months. However,
as of December 2007, the draft assessment has not yet been provided to
the National Academies. After verbally agreeing with both the noncancer
and cancer assessments following briefings on the assessments, the
Assistant Administrator, Office of Research and Development,
subsequently requested that additional uncertainty analyses--including
some quantitative analyses--be conducted and included in the assessment
before the draft was released to the National Academies for peer
review. As discussed above, quantitative uncertainty analysis is a risk
assessment tool that is currently being developed, and although the
agency is working on developing policies and procedures for uncertainty
analysis, such guidance currently does not exist. The draft
tetrachloroethylene assessment has been delayed since early 2006 as EPA
staff have gone back and forth with the Assistant Administrator trying
to reach agreement on key issues such as whether a linear or nonlinear
model is most appropriate for the cancer assessment and how uncertainty
should be qualitatively and quantitatively characterized. EPA officials
and staff noted that some of the most experienced staff are being used
for these efforts, limiting their ability to work on other IRIS
assessments. In addition, the significant delay has impacted the
planned National Academies peer review because the current contract,
which has already been extended once, cannot be extended beyond
December 2008. The peer review was initially estimated to take 15
months. As a result, a new contract and the appointment of another
panel may be required.
Dioxin. The dioxin assessment is an example of an IRIS assessment that
has been, and will likely continue to be, a political as well as a
scientific issue. Often the byproducts of combustion and other
industrial processes, complex mixtures of dioxins enter the food chain
and human diet through emissions into the air that settle on soil,
plants, and water. EPA's initial dioxin assessment, published in 1985,
focused on the dioxin TCDD (2,3,7,8-tetrachlorodibenzo-p-dioxin)
because animal studies in the 1970s showed it to be the most potent
cancer-causing chemical studied to date. Several years later, EPA
decided to conduct a reassessment of dioxin because of major advances
that had occurred in the scientific understanding of dioxin toxicity
and significant new studies on dioxins' potential adverse health
effects. Initially started in 1991, this assessment has involved
repeated literature searches and peer reviews. For example, a draft of
the updated assessment was reviewed by a scientific peer review panel
in 1995, and three panels reviewed key segments of later versions of
the draft in 1997 and 2000. In 2002, EPA officials said that the
assessment would conclude that dioxin may adversely affect human health
at lower exposure levels than had previously been thought and that most
exposure to dioxins occurs from eating such American dietary staples as
meats, fish, and dairy products, which contain minute traces of
dioxins. These foods contain dioxins because animals eat plants and
commercial feed and drink water contaminated with dioxins, which then
accumulate in animals' fatty tissue. It is clear that EPA's dioxin risk
assessment could have a potentially significant impact on consumers and
on the food and agriculture industries. As EPA moved closer to
finalizing the assessment, in 2003 the agency was directed in a
congressional appropriations conference committee report to not issue
the assessment until it had been reviewed by the National Academies.
The National Academies provided EPA with a report in July 2006. In
developing a response to the report, which the agency is currently
doing, EPA must include new studies and risk assessment approaches that
did not exist when the assessment was drafted. EPA officials said the
assessment will be subject to the IRIS review process once its response
to the National Academies' report is drafted. As of 2008, EPA has been
developing the dioxin assessment, which has potentially significant
health implications for all Americans, for 17 years.
While Appropriate Coordination with Federal Agencies Could Help EPA
Resolve IRIS Assessment Controversies More Efficiently, EPA's Proposed
Expansion of Agencies' Roles in IRIS Assessments Would Cause Further
Delays and Limit Their Credibility:
Although an interagency review process managed by OMB was informally
incorporated into the IRIS assessment starting in 2004, federal
agencies continue to believe they should have greater and more formal
roles in EPA's development of IRIS assessments given the potential
impact of the assessments on either their missions or their budgets,
such as the need to redesign systems to eliminate hazardous materials
or to clean up contaminated sites. These agencies--including DOD, DOE,
the Department of Homeland Security, the Department of Transportation,
and NASA--have sought earlier, more formal involvement in IRIS
assessments than is currently provided through the OMB/interagency
review process and the other avenues for input that currently exist,
which include nominating chemicals for assessment and providing
relevant studies for planned or ongoing assessments. Officials from
DOD, NASA, and DOE told us there is a lack of formality and
transparency about how they can provide input and when it is
appropriate to do so. For example, they seek a more formal process for
nominating chemicals and providing relevant studies for planned or
ongoing assessments, and they want to help decide questions for the
peer reviews of IRIS assessments. They said that reducing the
scientific uncertainty in IRIS assessments is important because some
assessments can have significant impacts on their operations and
budgets. Also, the officials said that their involvement with IRIS
assessments occurs too late in the process, leading to disagreement
among EPA and the agencies after the assessments are drafted, and
unnecessarily delaying certain assessments, such as naphthalene, TCE,
and dioxin.
Along these lines, we concluded in a 2006 report that EPA could help
ensure consistent, transparent, and high-quality risk assessments by
working with stakeholders early and periodically throughout the process
to identify, among other things, key issues and studies that need to be
considered in the analysis.[Footnote 37] Nonetheless, we believe that
if EPA is to increase the involvement of DOD and other potentially
affected federal agencies in the IRIS assessment process, it is
important to do so in a way that enables EPA to balance the benefits of
increased interagency coordination with EPA's need to improve the
timeliness of its IRIS assessments and to ensure their credibility with
proper controls.
In response to the continuing concerns of some federal agencies
regarding the IRIS assessment process, EPA has, for several years, been
working to develop a formal IRIS assessment process that would give
other federal agencies a more significant role in the process.[Footnote
38] EPA consulted with an Interagency Work Group on the IRIS Process,
which includes officials representing potentially affected agencies
such as DOD, DOE, and NASA, as well as officials from other agencies
that develop health assessments of chemicals, such as the Department of
Health and Human Services' ATSDR. OMB, the Department of Homeland
Security, and the White House Office of Science and Technology Policy
are also represented on the working group.
EPA developed a draft process in early 2007 that the agency believed
would be acceptable to the interagency work group. One of the key
aspects of EPA's draft process is an increased role for other agencies-
-including those whose operations and resource levels could be affected
by the results of the assessments--in providing input to IRIS chemical
assessments. Specifically, federal agencies would have the opportunity
to be involved, or provide some form of input, at almost every step of
EPA's IRIS assessment process. Most significantly, the draft proposed
process would provide federal agencies with several opportunities
during the IRIS assessment process to identify chemicals of interest to
them as "mission critical." As defined in the draft process, a mission-
critical chemical is one that is "an integral component to the
successful and safe conduct of an agency's mission in any or all phases
of operations." Potential impacts on the use of mission-critical
chemicals assessed under the IRIS program include "cessation or
degradation of the conduct of the mission and/or unacceptable resource
constraints."[Footnote 39] A mission-critical designation would add
requirements to the assessment process, providing other federal
agencies with increased involvement in IRIS assessment decisions. As
outlined, this designation could add 2 or more years to the process
(see shaded boxes in fig. 5 for these additional requirements).
Figure 5: EPA's Draft Proposed IRIS Assessment Process:
[See PDF for image]
This figure is an illustration of EPA's Draft Proposed IRIS Assessment
Process, as follows:
1. Annual request for nomination of IRIS chemicals for assessment[A].
2. Determine the annual agenda and publish it in the Federal
Register[A].
3. Scientific literature search.
4. Federal Register notice/data call-in[A].
5. Development of a draft qualitative assessment. Is the chemical
mission critical? If yes, go to step 6; if no, go to step 10.
6. Public review of draft qualitative assessment[B].
7. Review of public comments[B].
8. Interagency evaluation to close data gaps[B]. Is there interest in
conducting research to close data gaps? If yes, go to step 9; if no, go
to step 10.
9. Federal agency development of new studies[B].
10. Development of a complete draft IRIS assessment (qualitative and
quantitative), including internal peer consultation. Is the chemical
mission critical? If yes, go to step 11; if no, go to step 12.
11. EPA consults with federal agencies to determine level and scope of
peer review[B]. Go to step 13.
12. EPA determines the level and scope of the peer review.
13. Internal agency review by the 18-member IRIS Agency Review
Committee.
14. Revised draft IRIS assessment. Return to step 13 if necessary.
15. OMB/interagency review coordinated by OMB[C].
16. Revised draft IRIS assessment. Return to step 15 if necessary.
17. Independent external peer review and public comment[C].
18. Revised draft IRIS assessment.
19A. Second internal agency review by the 18-member IRIS Agency Review
Committee. Return to step 18 if necessary.
19B. Second OMB/interagency review coordinated by OMB[C]. Return to
step 18 if necessary.
20. Completion of IRIS assessment, EPA management review and approval,
and posting on IRIS.
[A] Additional steps, under EPA’s planned changes, to its assessment
process and indicate steps where EPA has provided additional
opportunity for input from potentially affected federal agencies for
mission-critical chemicals.
[B] Indicate steps where EPA has provided additional opportunity for
input from potentially affected federal agencies for all chemicals.
[C] Indicate steps where potentially affected federal agencies already
had an opportunity for input.
Source: GAO analysis of EPA information.
[End of figure]
The additional requirements for assessments of mission-critical
chemicals that would lengthen the IRIS assessment process include the
following:
* The opportunity for federal agencies and the public to identify any
additional information that is available on a chemical and to correct
any errors on EPA's new first assessment draft--a draft qualitative
assessment.[Footnote 40] EPA estimates that the public comment period
alone would add 45 days to the assessment process.
* An opportunity for potentially affected federal agencies to review
public comments made during the error correction step and initiate a
meeting with EPA if they want to discuss a particular set of comments.
EPA estimates that this step, which includes EPA's review and analysis
of the public comments, would add 30 days to the process.
* An opportunity for potentially affected federal agencies to fill a
data gap or eliminate an uncertainty factor that is identified in the
qualitative draft. If an agency believes it can sponsor research to
fill a significant data gap, EPA would then suspend the assessment
process for up to 18 months to allow that agency time to conduct
research and have the completed study peer reviewed.
In addition, the draft proposed process would give other federal
agencies a formal role in helping EPA determine the level of
independent peer review assessments for mission-critical chemicals--
that is, whether the peer reviews would be conducted by EPA Science
Advisory Board panels, National Academies' panels, or panels organized
by an EPA contractor. In addition, the other federal agencies would be
able to help determine the panel members' areas of scientific expertise
as well as the scope of the peer reviews and the specific issues they
would address.
Finally, the draft proposed process would also formalize the roles of
other federal agencies in nominating chemicals for assessment, helping
EPA determine the chemicals it would assess, and providing scientific
information (reports, studies, etc.) for the assessments EPA
undertakes. In the past, EPA sought such information in public notices
in the Federal Register, and federal agencies could have provided their
input either in response to these notices or by contacting EPA
officials directly. However, in response to criticisms that EPA's
approach to seeking input from the public was not appropriate for other
federal agencies, under the proposed process, EPA would directly reach
out to other federal agencies for this input by separately
communicating these requests to them at the time it publishes its
notices in the Federal Register. The draft proposed IRIS process would
also formalize the OMB/interagency reviews, which provide other federal
agencies with the opportunity to provide comments on IRIS draft
assessments both before and after the draft assessments are provided to
external peer reviewers. The OMB/interagency reviews would continue to
be managed by OMB.
EPA has acknowledged that the additional steps and opportunities for
input from other federal agencies that its draft proposed process would
provide will add more time to an already lengthy process. Specifically,
under the proposed process we reviewed, EPA officials estimated that
IRIS assessments for standard chemicals would take roughly 2-1/2 years
to complete and that an additional 2 to 3 years would be needed for
mission-critical chemicals. In February 2008, the Assistant
Administrator, Office of Research and Development, updated these time
frames, estimating that most assessments would take between 3 and 4-1/
2 years to complete, while assessments of mission-critical chemicals
would take up to 6 years. However, these estimates appear to be
optimistic considering the length of time ongoing assessments--which
have not been subject to additional process steps--have been in
progress: most for more than 5 years and many for more than 9 years.
Further, when assessments take longer than 2 years, they can become
subject to substantial delays stemming from the need to redo key
analyses to take into account changing science and assessment
methodologies.
According to officials from EPA and some of the potentially affected
federal agencies, the key goals of the draft proposed changes to the
IRIS assessment process are to (1) fill data gaps to reduce uncertainty
in the IRIS assessments and (2) prevent disagreements among EPA and
other federal agencies about assessments late in the IRIS assessment
process. As discussed earlier, disagreements have arisen late in the
process in the past and have significantly delayed the completion of
some assessments. These goals are important, and it is appropriate for
EPA and the federal agencies to find ways to achieve them. However,
some of the proposed changes would result in assessments being caught
in a continuous cycle of updates and revisions--delaying the completion
of assessments of chemicals with potentially significant health impacts
for many years. As discussed above, delays for any reason--including
waiting for new research to fill data gaps--can have a compounding
effect on time frames because delays often require extensive rework,
such as updates to reflect other studies that have come out in the
interim, performing additional analyses to reflect the current state of
the science of risk assessment tools, and other rounds of interagency
or external peer reviews because of changes in the assessments. Thus,
changing from a reliance on the best available scientific data at the
time an assessment is conducted to a reliance on research that has
either not yet started or been completed in order to fill scientific
data gaps is a decision that can have a significant impact on
assessment completion dates.
Further, although EPA has said that IRIS assessments would only be
suspended to wait for shorter-term studies, such as those assessing
modes of action, we and several agency officials we spoke with believe
that the time needed to plan, conduct, and complete research that would
address significant data gaps, and have it peer reviewed, would likely
take longer than the 18 months EPA would allow under its draft proposed
IRIS process. In addition, although the draft process would set a limit
on the time the assessments can be suspended to await studies, the
enforcement of such time frames could prove to be difficult when
research schedules slip. And not waiting for study results beyond the
18-month limit could, in fact, undo all of the efforts expended to
avoid disagreements in the later stages of the assessments. Also, EPA's
prior experience with waiting for studies, such as in the case of the
formaldehyde assessment discussed earlier, shows that estimates of
completion dates for these studies can stretch out from months to
years. Finally, the extent to which new research would provide
information that would actually reduce key scientific uncertainties in
the IRIS assessments is not known until the research is completed and
peer reviewed.
In discussing with us the need for additional time to conduct new
research to fill scientific data gaps for chemicals defined as mission
critical, DOD officials emphasized that the federal agencies believe it
is important that scientific uncertainties are reduced to the maximum
extent practical within a reasonable time frame before IRIS assessments
of mission-critical chemicals with high uncertainty are completed. The
potential impacts of IRIS assessments on DOD--should risk assessments
using the IRIS data lead to regulatory actions--could include the need
to (1) conduct research and development of material substitutes; (2)
redesign systems and processes to eliminate hazardous materials; (3)
conduct research and development of treatment or cleanup technologies;
(4) improve personal protective clothing, equipment, and procedures;
and (5) provide for special handling and storage of chemicals.
However, the reality is that scientific data gaps will almost always
exist. EPA, whose mission includes protecting the public and the
environment from harmful chemicals, must decide whether to use
assumptions and estimates to address data gaps or to wait for research
that could potentially fill some of the data gaps. On this issue, in
discussing the fact that EPA must address many environmental risks with
incomplete data and a lack of consensus about assumptions, the agency
has stated that it seeks to strike a balance among fairness, accuracy,
and efficient action because not acting until data quality improves can
result in substantial harm to human health, safety, and the
environment. Along these lines, as discussed earlier, a National
Academies panel recently reported on a chemical of interest
(trichloroethylene) to several federal agencies that would likely be
defined as mission critical under the proposed IRIS process. The
National Academies' report recommended that the risk assessment be
finalized with currently available data, rather than waiting for
additional information to address scientific uncertainties, so that
risk management decisions can be made expeditiously because the
evidence on risks to human health have strengthened. The National
Academies specifically noted that while some additional information
would allow for more precise estimates of risk, this information is not
necessary at this time for developing a credible risk assessment. EPA
started the assessment in 1998, and it has been subject to multiple
independent peer reviews. However, EPA is currently preparing another
assessment draft that will need to undergo the IRIS assessment process
a second time. As such, the draft will be to subject to the OMB/
interagency reviews and another independent peer review before it can
be finalized. We note that if EPA's proposed IRIS process is
implemented, DOD and other federal agencies could designate
trichloroethylene as a mission-critical chemical and have the
assessment suspended for 18 months for research aimed at reducing
scientific uncertainties.
EPA could potentially minimize or eliminate requests to suspend ongoing
assessments to fill in scientific data gaps by better coordinating with
DOD's emerging contaminants program--established in 2006--that leads
and supports the Materials of Emerging Regulatory Interest Team
(MERIT).[Footnote 41] The intent of the emerging contaminants program
is to help DOD proactively manage chemicals DOD uses or has used in its
operations that (1) are a perceived or real threat to human health or
the environment and (2) lack a published health standard and/or have
new information available about their sources or effects. The program
seeks to identify chemicals of importance to DOD's mission that are on
the regulatory horizon so that the agency can begin to make changes or
identify areas of research that may minimize the impact that
regulations of a particular chemical would have on the agency and its
mission.
Of particular relevance to the IRIS program is the scanning component
of the emerging contaminants program, which consists of a regular
review of the scientific literature, periodicals, and regulatory
communications for chemicals that have, or are likely to have, changing
human health values or regulatory standards. DOD then further screens
those chemicals that are, were, or will be used by DOD to determine
whether a potential impact to DOD's mission exists. After screening,
DOD conducts impact assessments to determine the severity of risk to
DOD functional areas, which include environment, safety, and health
protection; research and development; production, operation and
maintenance, and disposal; training and readiness; and cleanup. Through
these efforts, DOD seeks to identify cost-effective opportunities to
proactively manage future unacceptable risks. One research area the
emerging contaminants program has identified as a potentially good
investment is DOD-funded health effects research aimed at filling
important data gaps for mission-critical chemicals with high
uncertainty factors.[Footnote 42]
DOD's ongoing actions in monitoring scientific literature and
identifying research that could reduce scientific uncertainties could
be more effectively put to use. Specifically, enhanced coordination and
collaboration between EPA and DOD's emerging contaminants program have
the potential to proactively reduce some scientific uncertainties
associated with chemicals that EPA is not yet assessing--as opposed to
the current reactive focus on ongoing IRIS assessments. That is, under
the draft proposed IRIS process changes we reviewed, DOD and other
federal agencies would be able to designate ongoing chemicals as
mission critical at several stages in the assessment process--and have
their assessments suspended for up to 18 months to address data gaps.
However, enhanced coordination and collaboration between DOD and EPA
could change this essentially reactive approach to a model that is more
proactive and effective. If, for example, EPA provided 2 years' notice
of its intent to assess specific chemicals, rather than simply
announcing the list of chemical assessments it plans to initiate during
a given fiscal year, DOD and other agencies could use this time to
sponsor and complete the studies they deem necessary, rather than
waiting to initiate them after an IRIS assessment had started. Giving
agencies 2 years' notice (which would provide them with more than 18
months to conduct additional research) would be more effective and
efficient than delaying ongoing IRIS assessments for 18 months to await
research, given the compounding effect of delays on IRIS assessments.
We note that, to date, the emerging contaminants action list--the list
of chemicals that DOD has assessed and judged to have a significant
potential impact on people or the DOD mission--has focused largely on
chemicals that EPA's IRIS program is already assessing. However, with
some adjustments in the emerging contaminants program's focus or scope,
DOD could widen its scope and help EPA with its planning process by
earlier identification of the mission-critical chemicals of concern to
DOD for which IRIS assessments are needed. Finally, when the emerging
contaminants program identifies health research needs for chemicals
considered mission critical, DOD could meet its research needs more
effectively and quickly by sponsoring research as needs are identified,
rather than waiting until IRIS assessments are started.
Lastly, we note that while increased coordination with potentially
affected federal agencies on IRIS assessments could enhance the
assessments and facilitate their completion, giving potentially
affected federal agencies formal roles in some IRIS assessment
decisions reduces the credibility of the assessments if proper
controls, such as ensuring transparency, are not in place. While we
recommended in our 2006 report on human health risk assessment that EPA
consistently involve stakeholders as appropriate to the risk
assessment, we made this recommendation in the context of improving the
overall quality, consistency, and transparency of risk assessments.
However, one aspect of EPA's draft proposed IRIS process that has
proven to be controversial involves transparency. Specifically, EPA's
early 2007 draft proposal included making the comments from OMB and
other federal agencies provided during the OMB/interagency review
process part of the public record; according to EPA, this requirement
has been removed from the latest draft proposal because of concerns
raised by OMB. These concerns have delayed the implementation of the
planned process changes, and as of December 2007, EPA was planning to
send a revised draft back to the interagency work group for review and
approval.
According to DOD officials, OMB did not support certain aspects of the
draft process EPA presented to the Interagency Work Group on the IRIS
Process in early 2007, expressing concerns about how the proposal would
address the concerns of states, industry, and environmental entities
and the requirement to have agencies' comments part of the public
record. The latter concern stemmed from the precedent it might set and
because OMB believes that doing so would erode the effectiveness of the
deliberative process among EPA and the agencies. However, because the
agencies' comments on IRIS assessments are to be scientific in nature,
it is unclear why agencies might not candidly comment on an IRIS
assessment even if their comments would become part of the public
record. In fact, officials from DOD, DOE, and NASA told us they did not
object to EPA's plan to make interagency comments part of the public
record because they would be scientific in nature and not part of a
policy discussion.
While OMB officials would not comment directly on this issue as it
relates to objections they might have to EPA's draft proposed process,
one official told us that generally OMB believes that effective
deliberations among federal agencies are important and that if
agencies' deliberative comments are part of the public record, agency
officials will not be as frank and candid as they would be under the
protection of confidentiality.
However, transparency in the IRIS assessment process can provide
assurance that these scientific assessments are appropriately based on
the best available science and that they are not impacted by policy
issues and considerations. Under the National Academies' risk
assessment and risk management paradigm, policy considerations are
relevant in the risk management phase, which occurs after the risk
assessment phase that encompasses IRIS assessments.[Footnote 43] Some
of the federal agencies that would be given formal roles in some IRIS
assessment decisions are particularly interested in risk management
issues as they would likely face adverse consequences, such as
increased cleanup costs and other legal liabilities, if EPA issues an
IRIS assessment for a given chemical that results in a risk management
decision to regulate the chemical to protect the public.[Footnote 44]
Consequently, to ensure credibility of the IRIS assessments, the input
of these agencies into the assessment process should be transparent.
Further, when EPA and other agencies propose regulations or other
actions on the basis of health risk assessments, the proposals
represent policy choices. Through the rule-making process and other
interagency working groups, federal agencies have the opportunity to
participate in policy dialogues. Such discussions can appropriately
address risk management concerns that, by definition, involve
integrating risk characterization information (based, in part, on
information in IRIS) with other information[Footnote 45] to decide how
to protect public health. In contrast, the input of federal agencies
into the IRIS assessments, part of the risk assessment process, should
be based solely on science issues, not policy concerns.
Conclusions:
The IRIS database, one of the most significant tools that EPA has
developed to effectively support its efforts to protect people and the
environment from harmful chemical exposures, is at serious risk of
becoming obsolete because the agency has not been able to keep its
existing assessments current or complete assessments of the most
important chemicals of concern. Although EPA has taken important steps
to improve the IRIS program and productivity since 2000 and has
developed a number of draft assessments for external review, its
efforts to finalize the assessments have been thwarted by a combination
of factors: the imposition of external requirements, the growing
complexity and scope of risk assessments, and certain EPA management
decisions. Each of these factors has led to delays in the completion of
individual IRIS assessments. In addition, an overarching factor--the
compounding effect of delays--has had a particularly profound impact on
productivity: Even a single delay can create a cascading series of
delays with far-reaching, time-consuming consequences. In fact, in some
cases, it is necessary to essentially start the assessment process over
because of the need to incorporate science and methodologies that have
evolved since the assessments began. To effectively meet its diverse
user needs, EPA must both keep the IRIS database up to date and
undertake assessments of potentially dangerous chemicals not yet
evaluated. Achieving these objectives will require EPA to complete IRIS
assessments in a timely fashion that minimizes rework--an outcome that
cannot occur unless EPA's IRIS assessment process is streamlined to
routinely support timely completion of assessments. When assessments
take longer than two years, they can become subject to substantial
delays stemming from the need to redo key analyses to take into account
changing science and assessment methodologies.
However, the manner in which other federal agencies are involved in the
IRIS assessment process prevents EPA from effectively streamlining and
managing its assessments. Under the current process, EPA is required to
send draft assessments to OMB at two key points in the assessment
process. Although the OMB/interagency reviews have no time frames or
deadlines, EPA is not allowed to proceed with the assessment process
until OMB agrees that EPA has sufficiently responded to agencies'
comments; and OMB has also directed EPA to terminate five assessments.
Further, OMB's view that disagreements between EPA and other agencies
should simply be elevated to senior EPA executives for resolution is
inefficient and causes delays. That is, an effective IRIS assessment
process would not require the constant intervention of top-level
executives, whose time is already at a premium. Moreover, the current
process elevates the goal of reaching interagency agreement above
achievement of IRIS program objectives and, in doing so, fails to
adequately acknowledge the expertise of EPA scientists and the many
other scientific experts who have prepared and reviewed the
assessments. We believe that interagency coordination can enhance the
quality of EPA's IRIS assessments. However, this goal would more likely
be achieved if the input from other federal agencies was obtained in a
manner that better ensured that EPA's scientific analyses were given
appropriate weight and that provided time frames to avoid indefinite
delays. Because an integral part of EPA's mission is to assess the
risks associated with exposures to chemicals, the agency is best
situated to establish how--and at what level--to best resolve IRIS
assessment issues raised by other federal agencies. Until EPA can
establish time frames for various assessment steps--including the OMB/
interagency review process--and determine (1) how it will resolve
interagency conflicts, (2) when assessments are ready for either
independent peer review or completion, and (3) the scope of the
assessments needed to support EPA programs, EPA will continue to be
hampered in its ability to develop timely and credible IRIS assessments
that meet its needs and protect the environment.
Importantly, the lack of transparency of the OMB/interagency review
process reduces the credibility of EPA's IRIS assessments. Because the
agencies' comments and the changes EPA makes in response are treated as
internal executive branch documents not subject to release outside the
executive branch, the OMB/interagency reviews occur in what amounts to
a black box. Specifically, the first assessment drafts that become
publicly available--those provided to independent peer reviews--
incorporate changes from federal agencies that may be affected by the
assessments without identifying or providing reasons for the changes.
Moreover, the changes EPA is proposing to its current assessment
process would provide potentially affected federal agencies with a
formalized opportunity to provide input at almost every step in the
assessment process without addressing the extent to which agencies'
input would be made part of the public record. Given the importance and
sensitivity of IRIS assessments, it is critical that input from all
parties, particularly agencies that may be affected by the outcome, be
publicly available to alleviate concerns of potential bias in the
assessments.
In addition, EPA's draft proposed changes to its IRIS assessment
process--which would further expand the role of other federal agencies
in the process, including the option to suspend assessments of key
chemicals for up to 18 months to develop new studies--would add to the
already unacceptable level of delays in completing IRIS assessments.
Under its draft proposed changes, EPA estimates that assessments of
certain key chemicals may take up to 6 years. Assessments of this
duration are problematic--they tend to get caught in a perpetual cycle
of updates and revisions. Further, we believe EPA's time estimate for
assessments under its draft proposal are optimistic, as most IRIS
assessments currently in progress have already been in the works for
more than 5 years, including 12 that have been in progress for more
than 9 years. Yet these assessments have not been subject to the
additional steps and requirements under the draft proposed process.
An alternative, more efficient approach to suspending assessments while
waiting for new research would be for EPA to give agencies and the
public more advance notice of planned assessments, thereby providing
external parties with an interest in conducting additional research on
a given chemical the ability to complete their work prior to the start
of the IRIS assessment. This is important because effectively
maintaining the IRIS database depends on strict adherence to time
frames using the best available science. In addition, we note that OMB
has raised questions about whether the proposed process sufficiently
addresses the needs of private entities. Along these lines, we believe
that if EPA grants special rights to other federal agencies to suspend
IRIS assessments to conduct new research, it is likely that the agency
would face pressure to do so for numerous other entities, including
industries and individual companies that could be impacted by IRIS
assessments should the assessments lead to regulatory actions.
Further, while we believe it is appropriate for EPA to identify key
uncertainties in IRIS assessments and to continue to work on developing
methodologies for conducting more sophisticated and meaningful
uncertainty analyses, continuing to delay IRIS assessments in order to
develop and test enhanced methodologies and strategies for
communicating them can conflict with EPA's stated goal of seeking
balance among fairness, accuracy, and efficient action. More
specifically, EPA has acknowledged that not acting until data quality
improves can result in substantial harm to human health, safety, and
the environment. Also, as EPA continues to incorporate uncertainty
analysis in its assessments, ensuring that the information is clear and
useful to decision makers is important. Along these lines, the National
Academies and others have warned that producing "ranges of meaningless
and confusing risk estimates" could result in assessments of reduced,
rather than enhanced, quality and objectivity.
Lastly, while it is difficult to overstate the importance of the IRIS
program to EPA's ability to effectively conduct its mission of
protecting human health and the environment, this program currently
uses about 0.1 percent of EPA's annual appropriations--specifically, in
fiscal year 2007, the program received about $9.6 million of EPA's $7.3
billion budget. EPA's current estimate that it will be able to complete
16 assessments a year by 2011 would represent a substantial increase
over recent productivity; however, it is not clear that this level will
be sufficient to maintain the viability of the IRIS database.
Recommendations for Executive Action:
To develop timely chemical risk information that EPA needs to
effectively conduct its mission, we are recommending that the
Administrator, EPA, require the Office of Research and Development to
re-evaluate its draft proposed changes to the IRIS assessment process
in light of the issues raised in this report and ensure that any
revised process:
* clearly defines and documents a streamlined IRIS assessment process
that can be conducted within time frames that minimize the need for
wasteful rework and carefully considers the trade-off between the
benefits of changes that involve additional steps and time--including
the development of enhanced uncertainty analyses and other emerging
assessment methods--and the impacts of these changes on EPA's ability
to complete timely chemical assessments;
* establishes a policy that endorses conducting IRIS assessments on the
basis of peer-reviewed scientific studies available at the time of the
assessment and develops criteria for allowing assessments to be
suspended to await the completion of scientific studies only under
exceptional circumstances;
* establishes IRIS assessment needs to provide at least 2 years' notice
of assessments that are planned, including criteria for making
exceptions to the advance notifications, if needed;
* sets time limits for all parties, including OMB and other federal
agencies, to provide comments to EPA on draft IRIS assessments; and:
* periodically assesses the level of resources that should be dedicated
to this significant program to meet user needs and maintain a viable
IRIS database.
In addition, we recommend that the EPA Administrator take steps to
better ensure that EPA has the ability to develop transparent, credible
IRIS chemical assessments--an ability that relies in large part on
EPA's independence in conducting these important assessments. Actions
that are key to this ability include ensuring that EPA:
* can determine the types of IRIS assessments to conduct on the basis
of the needs of EPA's program offices and other users;
* can define the appropriate role of external federal agencies in EPA's
IRIS assessment process and manage an interagency review process in a
manner that enhances the quality, transparency, timeliness, and
credibility of IRIS assessments, including determining when interagency
issues have been appropriately addressed; and:
* has the ability to provide comments by OMB and other federal agencies
on draft IRIS assessments to decision makers, the Congress, and the
public.
Agency Comments and Our Evaluation:
In commenting on a draft of this report, EPA's Assistant Administrator
for Research and Development agreed to consider our recommendations in
revising the IRIS assessment process. However, EPA stated that it
believed the productivity and transparency issues discussed in the
draft report were misrepresented in the title and body of the report.
We disagree and believe we have fairly represented IRIS productivity
and transparency issues related to the IRIS assessment process. We did
clarify that the transparency issues highlighted in our report focus on
the IRIS assessment process rather than on the content of IRIS
assessments, and we revised the report title. In addition, EPA
emphasized that the proposed changes to the IRIS assessment process are
still subject to change. Along these lines, we made our recommendations
with the intent that they would be integrated into any revised process.
In its response, EPA also estimated that under the new process, most
assessments would take between 3 and 4-1/2 years; mission-critical
assessments would take up to 6 years. These estimates differ from the
time frames EPA officials provided during our review, and we have
revised the report to reflect this. However, we believe an IRIS
assessment process built around such time frames is problematic. As we
state in our draft and final reports, when assessments take longer than
2 years, they can become subject to substantial delays stemming from
the need to redo key analyses to take into account changing science and
assessment methodologies. Finally, EPA asserted that the new process
will lead to increased productivity. We disagree and have concluded
that the proposed revisions would add to the already unacceptable level
of delays in completing IRIS assessments.
In its comments, OMB did not specifically address the recommendations
we made to EPA but disagreed with some aspects of the report, primarily
regarding our characterization of the OMB-managed interagency reviews
and their effects on IRIS assessments. For example, OMB indicated that
it disagreed with our conclusions that the OMB/interagency reviews make
it more difficult for EPA to complete IRIS assessments in a timely
manner, that these reviews affect the credibility of the assessments,
and that interagency comments should be transparent. We disagree with
OMB and believe that we have fairly represented the OMB/interagency
review process as well as the importance of input from all parties
being publicly available. Given the importance and sensitivity of IRIS
assessments, it is critical that input from all parties, particularly
agencies that may be affected by the outcome, be publicly available to
alleviate concerns of potential bias in the assessments. EPA's and
OMB's comments and our detailed responses appear in appendix IV and
appendix V.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution of this report
until 30 days from the report date. At that time, we will send copies
to the Administrator, EPA; the Director, OMB; and appropriate
congressional committees and other interested parties. We will also
make copies available to others on request. In addition, the report
will be available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-3841 or stephensonj@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major contributions
to this report are listed in appendix VI.
Sincerely yours,
Signed by:
John B. Stephenson:
Director, Natural Resources and Environment:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
This appendix details the methods we used to assess the Environmental
Protection Agency's (EPA) management of its Integrated Risk Information
System (IRIS). For this review, we determined (1) the outcome of steps
that EPA has taken to ensure that IRIS contains current, credible
chemical risk information, to address the backlog of IRIS assessments,
and to respond to new requirements from the Office of Management and
Budget (OMB); and (2) the potential effects of EPA's planned changes to
the IRIS assessment process on EPA's ability to ensure that IRIS
provides current, credible risk information.
To address these two objectives, we reviewed relevant EPA and other
documents, including EPA's 2003 IRIS Needs Assessment, the Office of
Research and Development's multiyear plans, numerous documents
outlining the IRIS assessment process and draft proposed changes to the
assessment process, EPA budget justification documents, and OMB's
Program Assessment Rating Tool assessment that covered the IRIS
program. We interviewed officials from EPA's National Center for
Environmental Assessment who manage the IRIS assessment program,
including the Center Director, the Associate Director for Health, and
the IRIS Program Director; officials from other EPA program offices,
including the Office of Air and Radiation; and officials from federal
science and health agencies that are involved in the IRIS assessment
process, to obtain their perspectives on, among other things, the
current IRIS assessment process, the potential effects of the draft
proposed changes to the process, the extent to which EPA has made
progress in completing assessments and meeting user needs, and
challenges EPA faces in completing assessments. In addition, we
interviewed officials from the Department of Defense, the Department of
Energy, and the National Aeronautics and Space Administration who have
served on the interagency working group on the IRIS assessment process,
and OMB officials in the Office of General Counsel and the Office of
Information and Regulatory Affairs to obtain their perspectives on the
OMB/interagency review process and on the planned changes to the IRIS
assessment process. We also interviewed officials from the Department
of Health and Human Services' Agency for Toxic Substances and Disease
Registry (ATSDR), an agency that assesses the potential noncancer
health effects of exposure to some chemicals, to obtain information on
ATSDR's assessment process. In addition, we attended the Board of
Scientific Counselors Human Health Risk Assessment (HHRA) Subcommittee
meetings in November 2007. During these meetings, the Subcommittee
reviewed the EPA Office of Research and Development's HHRA program--
specifically, its relevance, quality, performance, and scientific
leadership.
For the first objective, we obtained and analyzed several data sets,
including data from EPA's IRIS database; EPA's IRIS Track--a
compilation of status reports for IRIS assessments in progress; Federal
Register notices announcing annual IRIS agendas; a spreadsheet that EPA
uses to track the IRIS assessments sent to OMB for the OMB/interagency
reviews; EPA's 2003 screening level reviews report--its efforts to
identify assessments that may need to be updated; and EPA's 2003 IRIS
Needs Assessment. From the data we gathered, we analyzed information on
EPA's productivity, including the number of new and completed IRIS
assessments, for fiscal years 2000 through 2007; the status of IRIS
assessments, as of December 1, 2007, that were in progress during
fiscal year 2007; the status of IRIS assessments that have been sent to
OMB for OMB/interagency review and the time elapsed during those
reviews; the number of assessments in the IRIS database that may need
to be updated; the resources provided to the program for fiscal years
2000 through 2007; and user needs and EPA's assessment completion
goals. We also interviewed chemical managers, who are responsible for
managing the IRIS assessments in progress, to obtain further
information on the 77 chemicals in progress during fiscal year 2007. We
did not evaluate the scientific content or quality of IRIS assessments.
In addition, we conducted a reliability review of the data we received
from EPA for our first objective. Through our review, we determined
that the data we used were sufficiently reliable for our purposes. Our
assessment consisted of interviews with officials about the data system
and elements and the method of data input, among other areas. We also
corroborated the data with other sources, where possible. For example,
we verified the information provided in EPA's IRIS Track system with
the chemical managers responsible for each of the 77 chemicals in
progress during fiscal year 2007, and updated the information as
appropriate. Similarly, we ensured that status information presented in
IRIS Track was consistent with the status information presented in the
spreadsheet that EPA uses to track draft assessments provided to OMB
for the OMB/interagency review. In addition, EPA attempted to
corroborate its data on the status of IRIS assessments that have been
sent to OMB for OMB/interagency review and the time elapsed during
those reviews, providing OMB with its tracking spreadsheet for review.
OMB chose not to respond. Consequently, we relied on EPA's data and
assessed its reliability, based on information including the source of
the data and method of input.
We conducted this performance audit from October 2006 to March 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: Broad Outline of EPA's Current IRIS Assessment Process:
Note: This outline does not include the individual activities,
including internal reviews and briefings, encompassed within many of
the broad IRIS assessment categories.
1. Annual request for nomination of IRIS chemicals for assessment.
2. Determine the annual agenda and publish it in the Federal Register,
along with a call for scientific information from the public on select
chemicals.
3. Scientific literature search.
4. Development of a complete draft IRIS assessment (qualitative and
quantitative), including internal peer consultation.
5. Internal agency review by the 18-member IRIS Agency Review
Committee.
6. Revised draft IRIS assessment.
7. OMB/interagency review coordinated by OMB.
8. Revised draft IRIS assessment.
9. Independent external peer review and public comment.
10. Revised draft IRIS assessment.
11A. Second internal agency review by the 18-member IRIS Agency Review
Committee. Return to step 10 if necessary.
11B. Second OMB/interagency review coordinated by OMB. Return to step
10 if necessary.
12. Completion of IRIS assessment, EPA management review and approval,
and posting on IRIS.
Source: GAO analysis of EPA information.
Note: This outline does not include the individual activities,
including internal reviews and briefings, encompassed within many of
the broad IRIS assessment categories.
[End of section]
Appendix III: Information on OMB/Interagency Reviews of IRIS
Assessments:
As of December 1, 2007, EPA has sent 36 draft IRIS assessments to OMB
for the first OMB/interagency review managed by OMB, and the reviews of
16 assessments were completed. As shown in figure 6, the OMB/
interagency review of 4 assessments was completed in less than 6 months
and 12 of these assessments added 6 or more months to the IRIS
assessment process.
Figure 6: Status of IRIS Assessments Sent to OMB for the First OMB/
Interagency Review Starting in Fiscal Year 2004, as of December 1,
2007:
[See PDF for image]
This figure is a vertical bar graph depicting the following data:
Number of assessments:
Status: Completed interagency review in less than 6 months;
Number of assessments: 4.
Status: Completed interagency review in 6 or more months;
Number of assessments: 12.
Status: Assessment suspended, now being redone;
Number of assessments: 1.
Status: Assessment discontinued after a year in OMB/interagency review;
Number of assessments: 5.
Status: OMB/Interagency review ongoing for 6 months to a year;
Number of assessments: 1.
Status: "Interagency review ongoing for 2 to 4 months[A];
Number of assessments: 13.
Source: GAO analysis of EPA data.
[A] These 13 assessments were submitted to OMB in the last quarter of
fiscal year 2007.
[End of figure]
Moreover, as also shown in figure 6, five assessments that assessed
acute exposure were discontinued at the direction of OMB after a year
in the OMB/interagency review stage,[Footnote 46] and one assessment
was suspended and is now being redone (naphthalene). Finally, 14
assessments were at the OMB/interagency review stage as of December 1,
2007: Of these, 1 had been at this stage for 9-½ months, and the
remaining 13 had been submitted to OMB in the last quarter of fiscal
year 2007 and had been at the OMB/interagency review stage for 2 to 4
months. While OMB officials said that EPA does not have to provide a
document addressing every OMB/interagency comment, EPA officials said
that a detailed disposition of comments document was, in fact,
necessary in order to get OMB's agreement that EPA had satisfactorily
addressed all comments. There are no time frames for this iterative
process.
OMB officials said that the second OMB/interagency review is conducted
by OMB to ensure that EPA has adequately considered the comments from
the external peer review panel. As of December 1, 2007, 10 assessments
had been sent to OMB for the second OMB/interagency review, and 5 of
these assessments completed the review. The time frames for these five
assessments ranged from 10 days to almost 4 months.
[End of section]
Appendix IV: Comments from the Environmental Protection Agency:
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
United States Environmental Protection Agency:
Office Of Research And Development:
Washington, D.C. 20460:
[hyperlink, http://www.epa.gov]:
February 21, 2008
John B. Stephenson, Director:
Natural Resources and Environment:
Government Accountability Office:
441 G Street NW, Room 2135:
Washington, DC 20548:
Dear Mr. Stephenson:
Thank you for the opportunity to review and comment on the Government
Accountability Office's (GAO) draft report, "Low Productivity and Lack
of Transparency Limit the Usefulness and Credibility of EPA's
Integrated Risk Information System" (IRIS). Recognizing that EPA will,
in all probability, respond to the final version, I would like to
comment briefly on the productivity and transparency issues that are
discussed in the draft report, which we believe are misrepresented in
the report's title and throughout the report.
Since GAO raised many issues already recognized by EPA, the Agency has
been actively working on revisions to its IRIS process. We believe
these revisions will improve the productivity and transparency of IRIS.
It should be noted that although GAO discusses some of the aspects of
EPA's ongoing efforts to update the IRIS process, and comments on the
potential effects of the proposed changes to the process, the GAO draft
report refers to an outdated draft (internal) document. [See comment 1]
Consequently, some of the steps or procedures presented by GAO in its
draft report have changed or will no longer be relevant in the final
process. GAO's draft report should acknowledge that they were working
from a draft internal document, and therefore that GAO's conclusions
and recommendations may be erroneous when compared to the final IRIS
process. [See comment 2] Nonetheless, EPA will consider each of the
GAO's recommendations in light of the new IRIS process, even though
these recommendations were based on the outdated draft document.
Under the new process, EPA's increased involvement of other agencies
and the public will help us identify scientific issues earlier and
foster better communication and sharing of information, which will
ultimately help streamline the IRIS process. [See comment 3] It is
worth noting that under the new process, specific time limits for each
step in the process will improve the timeliness of completed
assessments. For example, public and interagency review periods will be
for fixed and relatively short durations. Interagency and public review
will therefore represent only a small fraction of the total time
required to complete an IRIS assessment. For most chemicals, the IRIS
process will take between three and four and a half years. For the
small number of mission-critical chemicals, the process may take an
extra one to one and a half years. EPA believes the new IRIS process,
in combination with the increased resources for the program provided in
recent years, will lead to increased productivity in achieving
completed assessments. [See comment 4]
Although only briefly recognized in GAO's draft report, today's IRIS
assessments are much more sophisticated, complex, and of higher
scientific quality than at any other time in the program's history.
[See comment 5] The increased complexity of chemical assessments often
leads to extensive public comments and more rigorous levels of internal
and external peer review, which is one of the most important reasons
for delays in the IRIS process. Additionally, recent external peer
reviews by EPA's Science Advisory Board (SAB) and the National Academy
of Sciences (NAS) have requested that even more sophisticated analyses,
such as quantitative uncertainty analysis, be performed and presented
in some IRIS assessments. These issues, and EPA's responsiveness to
them, are given too little attention or are unfairly criticized in the
GAO draft report. [See comment 6]
Also, the IRIS program has increased the rigor of its scientific peer
review. Specifically, the external peer review process has been changed
from letter reviews to face-to-face panel peer reviews that are open to
the public. This has encouraged public interaction with the peer
reviewers. Members of our Board of Scientific Counselors (BOSC) that
recently reviewed the IRIS program indicated that "the extent of peer
review for IRIS...exceeds most other examples with which the
subcommittee members were familiar." [See comment 7]
Transparency is critical to an effective IRIS process. It is equally
important that scientists and policymakers be able to have full and
frank discussions. Clearly, the quality of government decisions is
better and stronger when healthy skepticism is not discouraged. [See
comment 8] The current IRIS process strikes a balance between these two
principles by encouraging open discussion of science and science policy
questions, while appropriately protecting the deliberative process.
Enhancing transparency through these and other measures necessarily
means bringing many parties into the IRIS process. However, increased
participation does not diminish EPA's accountability as the final
decision-maker in establishing IRIS health values. All final IRIS
assessments must also stand on their own merit by undergoing public and
external peer review. It is noteworthy that in addition to having a
transparent process, it is important to ensure transparency in the
content of IRIS assessments (e.g., the rationale for relying on or
using specific data sets, assumptions, or models). The GAO draft report
does not comment on this facet of transparency, which directly bears on
the content and quality of IRIS assessments. [See comment 9]
We have taken a number of specific steps to enhance transparency in the
IRIS program. For example:
* We created IRIS Track on our web site to let everyone know the status
of every chemical being worked on by EPA. This step opened up the
process to all stakeholders, provided much greater transparency than
before, and enabled interested parties to follow the progress of an
assessment. [See comment 10]
* Final documents are now published on the IRIS public web site. This
includes all external public comments received and all the responses to
the comments. The entire record of the assessment available to the
public. [See comment 11]
* IRIS has expanded the nomination process for new assessments to
include a Federal Register Notice to allow the public to nominate
chemicals, as well as a direct request to EPA Offices and other federal
agencies asking for their nominations as well. [See comment 12]
Like GAO, the Environmental Protection Agency highly values the IRIS
Program, which stands as a model for: the international scientific
community of unbiased, public, external peer review; the application of
new scientific advancements in risk assessment; and focusing government
efforts on priority chemical risk issues. According to the most recent
review of the IRIS program by BOSC, "The comprehensiveness,
transparency, and consistency of the IRIS approach has made it into the
internationally-recognized standard in hazard characterization."
EPA is committed to continual enhancement of the IRIS Program, and in
that spirit, we appreciate your attention to the program. If you have
any comments or questions on our response, please contact me or Kevin
Teichman, Deputy Assistant Administrator for Science, at (202) 564-
6620.
Best regards,
Signed by:
George Gray:
Assistant Administrator:
The following are GAO's comments on the Environmental Protection
Agency's letter dated February 21, 2008.
GAO Comments:
1. Our analysis of the draft proposed changes to the IRIS assessment
process does not support EPA's assertion that the revised process EPA
is developing will improve the productivity and transparency of IRIS.
In fact, as discussed in our draft and final reports, the draft
proposed process would exacerbate existing productivity and
transparency concerns.
2. EPA emphasizes that our draft report includes information on
proposed revisions to the IRIS assessment process that "have changed or
will no longer be relevant" in the final version. Our draft and final
reports clearly indicate that we are providing information on EPA's
current assessment process as well as on proposed changes to the IRIS
process that EPA has been developing for years--but has not yet
finalized. For example, the draft and final reports state that "as of
December 2007, EPA was planning to send a revised draft back to the
interagency work group for review and approval." Moreover, in the draft
and final reports, the graphic outlining the proposed changes is titled
"EPA's Draft Proposed IRIS Assessment Process." Further, we made our
recommendations with the intent that they be considered and integrated
into any revised process that EPA finalizes. Finally, we note that EPA
did not identify any significant changes it has made to the proposed
process we outline in the draft and final reports, either when it
commented on the draft report or on a December 2007 statement of facts
provided to EPA for review for factual accuracy. However, to ensure
clarity, we have added "draft" prior to our use of the term "proposed
process" in a number of places.
3. EPA states that "under the new process," the increased early
involvement of other agencies and the public in identifying scientific
issues and sharing information will ultimately help streamline the IRIS
process. We note that other agencies and the public have the
opportunity for early involvement in IRIS assessments under the current
process. We do not believe that simply formalizing their involvement
will, in fact, streamline the process or improve the timeliness of
completed assessments.
4. We have revised the report to add the updated estimated time frames
for IRIS assessments under EPA's proposed revisions to the process
provided in EPA's letter. With most assessments estimated to take up to
4-1/2 years to complete, it appears unlikely that the IRIS program will
be able to produce quality assessments in a timely manner. As discussed
in our draft and final reports, when assessments take longer than 2
years, they can become subject to substantial delays stemming from the
need to redo key analyses to take into account changing science and
assessment methodologies. Further, although EPA states that only a
small number of IRIS assessments will be designated as mission-
critical, we note that the chemicals with this designation are likely
to be those posing widespread public health concerns that need to be
addressed expeditiously. As we state in the draft and final reports,
effectively maintaining the IRIS database will depend on strict
adherence to time frames using the best available science.
In addition, we believe that if EPA grants special rights to other
federal agencies to, among other things, suspend IRIS assessments to
conduct new research, it is likely that the agency would face pressure
to do so for other entities, including industries and individual
companies that could be impacted by IRIS assessments should they lead
to regulatory actions. Therefore, while EPA's stated intent is to limit
the number of mission-critical designations made by other federal
agencies, the number is likely to increase over time.
5. While EPA states that the draft report only briefly recognizes that
IRIS assessments are more sophisticated and complex, our draft and
final reports identify the growing complexity of risk assessments and
risk assessment methods and models as one of five key factors
contributing to EPA's inability to complete IRIS assessments in a
timely manner.
6. EPA states that external peer reviews have requested that more
sophisticated analyses, such as quantitative uncertainty analysis, be
performed and presented in some IRIS assessments. EPA further states
that its responsiveness to these requests is given too little attention
or is unfairly criticized in the draft report. In fact, our draft and
final reports clearly state that peer reviews of EPA's assessments have
sometimes recommended additional uncertainty analysis. Further, while
we conclude that it is appropriate for EPA to identify key
uncertainties in IRIS assessments and to continue to work on developing
methodologies for conducting more sophisticated and meaningful
uncertainty analyses, the methodologies are not yet developed. That is,
our draft and final reports indicate that EPA plans to release draft
reports in 2010 on methods for analyzing and characterizing uncertainty
in hazard and dose response and in physiologically based
pharmacokinetic models. Thus, we believe that continuing to delay IRIS
assessments to develop and test enhanced methodologies and strategies
for communicating them conflicts with EPA's stated goal of seeking
balance among fairness, accuracy, and efficient action. Specifically,
it is unclear how EPA's ongoing, nearly 2-year delay of the assessment
of tetrachloroethylene (perc)--a widespread groundwater contaminant--
pending the development of an acceptable uncertainty analysis
represents an appropriate balance between accuracy and efficient
action.
7. The draft and final reports discuss the changes EPA has made to its
IRIS assessment process, including the one highlighted in EPA's letter-
-a change from letter peer reviews to panel peer reviews open to the
public.
8. EPA states that the current IRIS process strikes a balance between
transparency and having full and frank discussions by encouraging open
discussion of science and science policy questions while appropriately
protecting the deliberative process. We disagree that such a balance
exists because the OMB/interagency review process is not transparent at
all. In its comments, EPA dismisses any impact of the opaque OMB/
interagency review process, emphasizing that all IRIS assessments
undergo public and external peer review. However, the presence of
transparency at a later stage of IRIS assessment development does not
excuse or explain its absence earlier. Further, the National Academies
have stated that the dialogue between risk assessors and risk managers
should not bias or otherwise color the risk assessment conducted, and
risk assessment and risk management activities should remain distinct.
Transparency in the IRIS assessment process can provide assurance that
these scientific assessments are appropriately based on the best
available science and that they are not impacted by policy issues and
considerations. Finally, because federal agencies' comments on IRIS
assessments provided via the OMB/interagency process are to be
scientific in nature, it is unclear why agencies would not candidly
comment on an IRIS assessment even if their comments would become part
of the public record.
9. EPA's comments note that transparency is applicable to both the
assessment process and the content of IRIS assessments (e.g., the
rationale for relying on or using specific data sets, assumptions, and
models). We have revised our report to recognize that transparency is
relevant to both the assessment process and the content of IRIS
assessments and to be clear that the transparency issues we discuss
primarily relate to the process. We note that many of the process
changes EPA has made beginning in the 1990s are aimed at improving the
transparency of the content of IRIS assessments as well as the process.
10. This information on EPA process improvements is provided in the
draft and final reports.
11. EPA stated that final IRIS documents are now published on the IRIS
public Web site and that the entire record is available to the public.
This statement is not accurate because the record of the OMB/
interagency review comments and EPA's responses to them are not made
available to the public.
12. The draft and final reports provide information on EPA's nomination
process for new assessments, which includes a Federal Register notice.
[End of section]
Appendix V: Comments from the Office of Management and Budget:
Executive Office Of The President:
Office Of Management And Budget:
Washington, D.C. 20503:
February 21, 2008:
Mr. John B. Stephenson:
Director, Natural Resources and Environment:
U.S. Government Accountability Office:
441 G Street, N.W., Room 2075:
Washington, D.C. 20548:
Dear Mr. Stephenson:
Thank you for the opportunity to comment on the Government
Accountability Office's (GAO) draft report titled, "Chemical
Assessments: Low Productivity and Lack of Transparency Limit the
Usefulness and Credibility of EPA's Integrated Risk Information System"
(GAO-08-440).
We applaud your efforts to evaluate the EPA Integrated Risk Information
System (IRIS) as we believe this is a very important EPA program. As
your report states, funding for the IRIS program has increased-from
$1.7 million in FY 2000 to $9.6 million in FY 2007; OMB has supported
changes that will improve the quality and efficiency of the IRIS
program.
However, the draft GAO report fundamentally misunderstands the purpose
of interagency review and, unfortunately, could leave readers with a
false impression of the role of OMB/interagency review in the IRIS
process. We discuss below our most important concerns with the draft
report. [Footnote 47] [See comment 2]
First, the draft report mischaracterizes OMB's role by alleging that
OMB has imposed an OMB-managed interagency review process of IRIS risk
assessment documents that has resulted in a loss of EPA control. [See
comment 1][Footnote 48] In fact, however, only EPA has the authority to
finalize an EPA assessment and only EPA has authority to determine when
a draft may be sent for external peer review. The interagency review
process is a dialogue that helps to ensure the quality (including
objectivity and transparency) of agency documents. Input from
scientists throughout the Federal government (including, for example,
HHS, DOD, DOE, NASA, DOL, and USDA within the context of the IRIS
process) helps inform and improve the quality of IRIS assessments.
These reviews address the objectivity, clarity, and transparency of the
work, identifying key science policy issues for EPA's consideration,
and scientific issues for external peer review. [Footnote 49] Simply
put, the interagency review process works as follows: OMB oversees an
interagency process that is part of a broader EPA process. During the
interagency process, EPA works collaboratively with OMB and other
agencies to achieve resolution and completion of interagency reviews.
These discussions have sometimes required resolution by the EPA Science
Advisor, who is also the EPA Assistant Administrator of the Office of
Research and Development. When this review process is complete, OMB
concludes interagency review. At that time, EPA moves to the next
step - generally, release of the risk assessment for public comment and
an external peer review by a group of independent experts. [Footnote
50] To be clear, EPA is entirely responsible for the content of
information on IRIS.
Second, the draft report alleges that the fact that OMB manages and
directs the review process has "made it more difficult for EPA to
complete IRIS assessments in a timely manner." [Footnote 51] [See
comment 3] The draft report asserts that the OMB/interagency review
adds significant time to IRIS time frames. It is our experience,
however, that the interagency commenters have generally provided
comments in a very timely manner. We are not aware of delays over
"nonsubstantive issues" as alleged in the draft report. [Footnote 52]
We also note that the draft report does not provide specific examples
on which we can comment. [Footnote 53] [See comment 6] In addition, we
believe the draft report is seriously flawed in that it does not
consider whether interagency review operates to improve the quality of
IRIS assessments, both by addressing the objectivity, clarity, and
transparency of the work as well as identifying key science policy
issues for EPA consideration and technical issues for peer review.
[Footnote 54] [See comment 4] The draft report's focus on timeliness,
without consideration of quality of the end product, presents a
misleading and incomplete picture.
Third, the draft report alleges that "the OMB/interagency review
process also affects the credibility of assessments primarily because
the review process lacks transparency." [Footnote 55] [See comment 5]
To address this supposed issue, the draft report recommends that EPA
make public comments by OMB and other Federal agencies to decision
makers, the Congress, and the public. However, the draft report fails
to acknowledge that, in the case of documents that involve interagency
deliberations, these documents are covered by the deliberative process
privilege, a well-recognized privilege that has been affirmed by the
Congress in the Freedom of Information Act (5 U.S.C. 552(b)(5)) and by
the Supreme Court in such cases as NLRB v. Sears, Roebuck & Co., 421
U.S. 132 (1975), and EPA v. Mink, 410 U.S. 73 (1973). [Footnote 56]
[See comment 8] Accordingly, protection of internal Executive Branch
communications is not "inconsistent with the principle of sound
science," contrary to the erroneous conclusions of the draft report.
[Footnote 57] In fact, the National Academy of Sciences, which is
recognized as a leader in scientific reviews, uses a multi-step review
process that includes deliberations that are withheld from the public.
[Footnote 58] [See comment 9]
Finally, the draft report discusses at some length a draft document
that it alleges serves to establish a "formal IRIS assessment process"
that EPA has been developing over the past two years. [Footnote 59]
[See comment 7] The discussion in the draft GAO report is misleading to
readers; EPA's draft document is not a final document. [Footnote 60]
[See comment 10] EPA has not completed the development of this process.
Indeed, the process will not be complete until EPA circulates its draft
to the public for comments and then releases a final product that is
responsive to those comments.
Thank you again for the opportunity to review and comment on your draft
report. We hope you take these comments into account in preparing your
final report.
Sincerely,
Signed by:
Kevin F. Neyland:
Deputy Administrator:
Office of Information and Regulatory Affairs:
The following are GAO's comments on the Office of Management and
Budget's letter dated February 21, 2008.
GAO Comments:
1. OMB states that the draft report mischaracterizes OMB's role by
alleging that OMB has imposed an OMB-managed interagency review process
of IRIS risk assessment documents that has resulted in a loss of EPA
control. We believe that we have appropriately described the OMB/
interagency review process and identified the time frame and assessment
content limitations it has placed on EPA. First, as discussed in the
draft and final reports, OMB has limited EPA's ability to determine the
types of assessments it will conduct to meet its program needs by
requiring EPA to terminate five IRIS assessments. Second, while EPA's
annual performance goal for IRIS assessments had been the number of
completed assessments, during the Program Assessment Rating Tool review
by OMB it was agreed that the number of assessments provided to OMB for
OMB/interagency review was the most appropriate annual measure of
performance because EPA "relinquishes direct control of production
dates" when it sends draft IRIS assessments to OMB. Third, our draft
and final reports are clear that EPA officially determines when drafts
are sent for external peer review or are finalized, but they are also
clear that prior to taking these steps, EPA must be notified by OMB
that EPA has adequately addressed interagency comments. Thus, the
current process elevates the goal of reaching interagency agreement
above achieving IRIS program objectives and in doing so fails to
adequately acknowledge the expertise of EPA scientists and the many
other scientific experts who have prepared and reviewed the
assessments. Importantly, the fact that the first assessment drafts
that become publicly available--those provided to independent peer
reviewers--incorporate changes from federal agencies that may be
affected by the assessments without identifying these changes or the
reasons they were made weakens the credibility of the assessments.
Finally, because the negotiations over OMB/interagency comments are not
disclosed, OMB's assertion that EPA is entirely responsible for the
content of information on IRIS is open to question.
2. GAO provides statements of facts to agencies to confirm the factual
accuracy of statements upon which reports are based. OMB commented that
we did not incorporate its concerns about the draft statement of facts
we provided to OMB. We considered OMB's comments that addressed factual
information and, in response, made some revisions as appropriate.
However, many of OMB's comments did not address facts but were focused
on broader issues, such as the scope and methodology of our review. We
disagreed with OMB's broad comments and thus did not make certain
changes OMB suggested. For example, OMB expressed concern that our
report discusses the development of proposed revisions to EPA's IRIS
assessment process, which has not been released for public comment. As
we discuss further in comment 10, GAO audits and reviews include those
providing prospective analyses of possible or planned agency actions.
3. OMB takes issue with our conclusion that the OMB/interagency review
has made it more difficult for EPA to complete IRIS assessments in a
timely manner. Our draft and final reports show that the initial OMB/
interagency reviews that had been completed as of December 1, 2007,
added 6 months or more to the IRIS assessment process. The average
length of time for these initial reviews was 7-1/2 months. Given the
impact of delays on IRIS assessments and the staff time needed to
respond to OMB/interagency comments, we believe such delays are
significant. Further, our report discusses five key factors that make
it difficult for EPA to complete assessments in a timely manner--one of
which is the OMB/interagency review process.
4. OMB states its belief that the draft report is seriously flawed
because it does not consider whether interagency reviews improve the
quality of IRIS assessments. As OMB notes, our draft report states that
we did not evaluate the scientific content or quality of IRIS
assessments. However, as the draft and final reports state, the Board
of Scientific Counselors--an independent board of experts appointed by
EPA to assess its programs--reviewed the effectiveness of the IRIS
program (as part of its review of EPA's human health risk assessment
program) in November 2007. A report is expected to be finalized in
early 2008. In response to our congressional request and to avoid
duplication of effort, we focused on the timeliness and credibility of
IRIS assessments in the context of the current assessment process and
proposed revisions to it. Our draft report also noted that it is too
early to determine whether or to what extent the IRIS assessment
process changes EPA has implemented in recent years have enhanced the
scientific credibility of the assessments--for example, only four
assessments were finalized in fiscal years 2006 and 2007. Importantly,
because OMB/interagency comments and EPA's response to them are not
disclosed, the extent to which the comments added value or caused EPA
to revise its risk estimates cannot be determined. However, the status
of the IRIS assessment of dibutyl phthalate raises questions about the
extent to which the OMB/interagency reviews improve the quality of IRIS
assessments. Specifically, this key assessment had cleared both OMB/
interagency reviews and was in the process of being finalized in July
2007. However, because of methodology concerns raised by the Natural
Resources Defense Council and the Chairman, Senate Committee on
Environment and Public Works, EPA suspended the assessment pending a
review by the National Academies.
5. OMB asserts that "the draft report fails to acknowledge that, in the
case of documents that involve interagency deliberations, these
documents are covered by the deliberative process privilege," and that
"accordingly, protection of internal Executive Branch communications is
not 'inconsistent with the principle of sound science.'" Contrary to
OMB's assertion, the report specifically acknowledges that OMB
considers the documents at issue to be protected from disclosure
because of their deliberative nature. Moreover, OMB's assertions
concerning the deliberative process privilege are misleading and
illogical. That is, OMB's comments fail to note that the deliberative
process privilege protects internal and interagency communications from
judicially compelled disclosure, an issue irrelevant to our
report.[Footnote 65] The privilege in no way prevents agencies from
voluntarily disclosing such information.[Footnote 66] OMB is thus
arguing that because the scientific comments at issue might generally
be protected from discovery in civil litigation, refusal to disclose
them voluntarily in this specific context is necessarily consistent
with the principles of sound science. OMB provides no citation or other
support for this conflation of judicial and scientific procedures.
Moreover, OMB's comments that the deliberative process privilege was
affirmed in the Freedom of Information Act (FOIA) fails to acknowledge
that FOIA requires federal agencies to disclose factual material in
documents covered by the deliberative process privilege if the factual
material can be reasonably segregated from the deliberative
material.[Footnote 67] OMB's comment appears to assume that because a
document passes from one agency to another all of its contents may be
withheld from disclosure, a legally unsupportable assertion that courts
have uniformly rejected, particularly in the context of attempts to
withhold scientific information.[Footnote 68] Indeed, to the extent
that interagency comments on IRIS assessments reflect political and
policy concerns rather than discussions of scientific issues, they
would be easier to withhold from disclosure in court but harder to
justify as part of the IRIS scientific assessment process.
In addition, OMB's comments do not distinguish between risk assessment
and risk management decisions, such as the formulation of regulations.
As discussed in the draft and final reports, IRIS assessments are not
themselves regulations, and OMB and other agencies will have
opportunities to engage in deliberative policy debates during, for
example, interagency reviews that occur during rule makings. These are
points in the process where it is appropriate to address policy
questions, such as the implications for other agencies of a specific
regulatory decision.
Finally, under the interagency process as currently contemplated, some
IRIS assessment reviewers--representatives of federal agencies--
essentially are given favored status. OMB fails to explain why certain
scientific views should be given added consideration and protected from
the critical scientific scrutiny all other comments will receive simply
because the reviewers providing the comments are federal employees.
6. OMB stated general disagreement with information presented in our
discussion of the naphthalene assessment but did not cite the specific
information with which it disagreed. Instead, OMB referred readers to
the charge and report from the second peer review EPA had conducted on
this chemical at OMB's direction. In our draft and final reports, we
report EPA officials' description of the purpose and conclusions of
this peer review. Specifically, EPA officials said (1) the agency
convened an additional expert review panel on genotoxicity to obtain
recommendations about short-term tests that OMB thought could be done
quickly and (2) the panel concluded that such research could not be
conducted in the short term. The questions posed to the peer reviewers
and the summary of results in the report cited by OMB are consistent
with EPA's description.
7. OMB says that the draft report is misleading to readers because
EPA's "draft document is not a final document." The report clearly
identifies the proposed IRIS assessment changes that EPA has been
working on. For example, the graphic outlining the proposed changes is
titled "EPA's Draft Proposed IRIS Assessment Process." The draft and
final reports also state that "as of December 2007, EPA was planning to
send a revised draft back to the interagency work group for review and
approval." However, to ensure clarity, we have added "draft" prior to
our use of the term "proposed process" in a number of places.
8. We did not assert that the OMB/interagency process is equivalent to
peer review, and we agree with OMB that this review process happens
before any external peer review occurs. However, because the OMB/
interagency process is opaque, neither peer reviewers nor the public
are privy to the changes EPA made to the draft assessments or the
charge questions to the peer review panels in response to the comments
from OMB and other federal agencies. The presence of transparency at a
later stage of IRIS assessment development does not excuse or explain
its absence earlier.
9. It is unclear why OMB attempts to rely on the Federal Advisory
Committee Act Amendments of 1997 to defend the lack of transparency in
the interagency IRIS review process. The transparency procedures
applicable to the National Academies' committees to which OMB refers
far exceed those that exist under the OMB/interagency IRIS process. For
example, under the legislation OMB cites, National Academies'
committees must (1) provide the names, biographies, and conflict of
interest disclosures of committee appointees; (2) provide an
opportunity for the public to comment on the proposed committee member
appointments; (3) ensure that meetings focused on data gathering are
generally open to the public; (4) provide the names of reviewers of
draft committee reports; and (5) provide summaries of any closed
committee meetings. 5 U.S.C. App. 2, § 15. The interagency portion of
the IRIS process does none of these things. Moreover, while the act
authorizes a National Academies' committee to close meetings at which
information exempt from disclosure under FOIA would be discussed, the
President of the National Academy of Sciences assured the bill's
sponsor, in a letter commenting on the House bill that would later be
enacted into law, that the Academy would not rely on the deliberative
process exemption as the basis for closing a meeting.
"I wish to assure you that we subscribe fully to the goal of providing
as much openness as possible in our work. In particular, we have no
intention of using Section 552(b)(5), which deals with interagency
memoranda, as a basis for closing meetings of Academy committees. In
fact, it is the Academy's standard practice not to treat the type of
material covered by Section 552(b)(5) as confidential input to any
Academy deliberative process. This procedure insures that, inasmuch as
possible, all the information that a committee uses to reach its
conclusion is in the public record."
10. We were asked by the Chairman, Senate Committee on Environment and
Public Works, to examine the potential effects of planned changes to
the IRIS assessment process on EPA's ability to ensure that IRIS
provides current, credible risk information. This review therefore
involved evaluating a draft of EPA's planned IRIS assessment changes.
GAO audits and reviews include those providing prospective analyses of
possible or planned agency actions. We may also assess the ability of
alternative approaches to yield better program performance or eliminate
factors that inhibit program effectiveness. As is typical in cases in
which we evaluate draft proposals, we make our recommendations on the
IRIS assessment process in the spirit of informing those revisions.
[End of section]
Appendix VI: GAO Contact and Staff Acknowledgments:
GAO Contact:
John B. Stephenson, (202) 512-3841 or stephensonj@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Christine Fishkin (Assistant
Director), Mark Braza, Nancy Crothers, Laura Gatz, Richard P. Johnson,
Summer Lingard, Susan Swearingen, and Delia Zee made key contributions
to this report. Also contributing to this report were Tim Bober, Phylis
Cline, Michael Derr, and Cynthia Taylor.
[End of section]
Footnotes:
[1] The National Academies comprises four organizations: the National
Academy of Sciences, the National Academy of Engineering, the Institute
of Medicine, and the National Research Council.
[2] Transparency is relevant to both the IRIS assessment process (for
example, the public availability of information about the assessment
process, the input from external reviews on draft assessments, and
EPA's responses to them) and the content of IRIS assessments (for
example, the rationale for using specific data sets, assumptions, or
models). In this report, the transparency issues we discuss primarily
relate to the IRIS assessment process.
[3] The cancer slope factor and unit risk typically are both upper
bound estimates (the plausible statistical upper limits of the true
value of a quantity).
[4] OMB, Fiscal Year 2006 Program Assessment Rating Tool (PART)
assessment of EPA's Human Health Risk Assessment Program.
[5] This internal peer review continues to be conducted during the
assessment drafting stage.
[6] OMB did not include these five assessments towards EPA's fiscal
year 2006 annual performance goals and thus does not acknowledge that
EPA sent 16 draft assessments to it for OMB/interagency review that
fiscal year.
[7] EPA has, in some cases, divided an ongoing assessment into two
assessments. For example, in fiscal year 2007, the arsenic assessment
covering both cancer and noncancer risks, which was started in 2003,
was divided into two separate assessments. Although the noncancer
assessment was started in 2003, IRIS Track, which tracks the status of
ongoing assessments, shows a 2007 start date for the IRIS noncancer
assessment.
[8] The two offices identified the same priority chemicals in eight
cases.
[9] The other nominations were for chemicals in IRIS with outdated
assessments.
[10] The preliminary findings of the Use of IRIS Project were presented
during the EPA's Board of Scientific Counselors' Human Health Risk
Assessment Subcommittee meeting on November 15, 2007.
[11] EPA, Human Health Risk Assessment Multi-Year Plan (Washington,
D.C., 2007).
[12] In its 2005 multiyear plan EPA, specified that it would send the
assessments for acrylamide, MTBE, naphthalene, tetrachloroethylene
(perc), acrylonitrile, formaldehyde, methanol, methylene chloride,
trichloroethylene, and dioxin for external review during fiscal years
2006 and 2007. Only acrylamide and tetrachloroethylene (perc) have been
sent to OMB for OMB/interagency review.
[13] Under the executive order, significant guidance is defined as, "a
guidance document disseminated to regulated entities or the general
public that may reasonably be anticipated to (1) lead to an annual
effect of $100 million or more or adversely affect in a material way
the economy, a sector of the economy, productivity, competition, jobs,
the environment, public health or safety, or State, local, or tribal
governments or communities; (2) create a serious inconsistency or
otherwise interfere with an action taken or planned by another agency;
(3) materially alter the budgetary impacts of entitlements, grants,
user fees or loan programs or the rights or obligations of recipients
thereof; or (4) raise novel legal or policy issues arising out of legal
mandates, the President's priorities, or the principles set forth in
this Executive order."
[14] Examples cited by the former director include scientific data such
as subtle biologic changes, biomarkers, and partial data on one or more
mechanistic hypotheses.
[15] Examples of PBPK model applications in risk assessments include
interspecies extrapolation of the dose-response relationship, route- to-
route extrapolation, estimation of response from varying exposure
conditions, estimation of human variability (within the whole
population or subpopulations), and high-to-low dose extrapolation.
[16] EPA plans to release draft reports in 2010 on methods for
analyzing and characterizing uncertainty in hazard and dose response
and in PBPK models.
[17] According to an EPA official, the November 2007 review of the IRIS
program by the Board of Scientific Counselors, an independent board of
experts appointed by EPA to assess its program, concluded that the
program "met expectations." A report covering this review, which
included evaluating the scientific quality of IRIS assessments, is
expected in early 2008.
[18] Also referred to as RDX or hexahydro-1,3,5-trinitrotriazine.
[19] The Board of Scientific Counselors is a federal advisory committee
established by EPA to provide advice, information, and recommendations
about the Office of Research and Development's research program.
[20] OMB recently proposed a bulletin that would have provided risk
assessment guidance to federal agencies. The proposed bulletin stated
that "every quantitative risk assessment should provide a range of
plausible risk estimates when there is scientific uncertainty or
variability." OMB decided not to finalize the bulletin after a National
Academies' committee severely criticized it, noting, among other
shortcomings, that "the description of uncertainty and variability in
the bulletin is oversimplified and does not recognize the complexities
of different types of risk assessments or the need to tailor
uncertainty analysis to a given agency's particular needs." In
particular, the committee noted that "a central estimate and a risk
range might be misleading in situations when sensitive populations are
of primary concern." National Academies, Scientific Review of the
Proposed Risk Assessment Bulletin from the Office of Management and
Budget Committee to Review the OMB Risk Assessment Bulletin
(Washington, D.C., 2007).
[21] In January 2006, the Assistant Administrator for Research and
Development was appointed as EPA's Science Advisor.
[22] EPA officials said that in July 2007, approval authority to send
draft assessments to OMB was again devolved to the Director, NCEA.
[23] According to the fiscal year 2004, 2005, and 2006 standard
operating procedures for IRIS assessments, before an assessment is
posted on the Web site, chemical managers are required to (1) brief the
IRIS Program Director and other NCEA management on any significant
changes since external review and (2) prepare a fact sheet for review
and approval by the Assistant Administrator, Office of Research and
Development, before the IRIS Program Director submits the final
assessment documents to the IRIS Web master for posting on the IRIS Web
site.
[24] A December 1998 notice stated that EPA planned to complete the
chloroform assessment, as well as 23 other assessments, in fiscal year
1999 or fiscal year 2000. In 2001, EPA completed assessments addressing
oral exposure to chloroform (oral RfD and cancer assessments); however,
the assessment addressing inhalation exposure (RfC) needed by the
Office of Air and Radiation is currently being drafted. Since 2002, EPA
has been reporting the chloroform (inhalation route) assessment as
being under way or generally complete and planned for entry into IRIS
within a year or 2.
[25] National Academies, Scientific Review of the Proposed Risk
Assessment Bulletin from the Office of Management and Budget Committee
to Review the OMB Risk Assessment Bulletin (2007).
[26] The European Union and California restrictions cover five
phthalates, including dibutyl phthalate.
[27] IRIS assessments cover individual chemicals, not cumulative
exposure to multiple related chemicals. This is an emerging risk
assessment issue that will require developing new assessment
methodologies. The congressional letter expressed concerns that the
draft IRIS assessment would not adequately protect public health but
did not request that it be entirely redone. Among other things, the
Natural Resources Defense Council letter suggested that EPA revisit its
decisions regarding the uncertainty factors used.
[28] Presentations at the Society for Risk Analysis meting reflect the
views of the authors and "do not necessarily reflect the views of any
other organization or agency."
[29] Using its 1996 Proposed Guidelines for Carcinogen Risk Assessment,
EPA concluded in the 1998 IRIS assessment of naphthalene that its human
carcinogenic potential could not be determined at that time, but noted
that there was suggestive evidence of potential human carcinogenicity.
(EPA also noted that under its 1986 cancer guidelines, EPA classified
naphthalene as a possible human carcinogen.) Subsequently, in 2002, the
International Agency for Research on Cancer (IARC), part of the World
Health Organization, concluded that naphthalene is possibly
carcinogenic to humans; in 2004, the Department of Human Health and
Services' National Toxicology Program concluded that naphthalene can
reasonably be anticipated to be a human carcinogen. EPA's current
assessment will be subject to the agency's 2005 cancer guidelines.
[30] According to DOD, EPA did not specifically ask the peer reviewers
to address some of the technical questions DOD had raised and wanted
the peer review to address.
[31] Genotoxic substances are a type of carcinogen, specifically those
capable of causing genetic mutation and of contributing to the
development of tumors. This includes both certain chemical compounds
and certain types of radiation.
[32] The National Priorities List is EPA's list of seriously
contaminated sites.
[33] The cancer portion of the formaldehyde assessment was originally
issued in 1989 and updated in 1991; the noncancer assessment was added
in 1990.
[34] NCI published the results of its study in two publications. The
first study, published in November 2003, focused on the association
between formaldehyde exposure and leukemia. M. Hauptmann, J. H. Lubin,
P. A. Stewart, R. B. Hayes, A. Blair, "Mortality from
Lymphohematopoietic Malignancies among Workers in Formaldehyde
Industries," Journal of the National Cancer Institute (2003). The
second study, published in June 2004, evaluated the association between
formaldehyde exposure and other cancers--including nasopharyngeal
cancer. M. Hauptmann, J. H. Lubin, P. A. Stewart, R. B. Hayes, A.
Blair, "Mortality from Solid Cancers among Workers in Formaldehyde
Industries," American Journal of Epidemiology (2004). The results of
the NIOSH study were described in one publication, dated March 2004,
which assessed mortality from all causes and all cancers. L. E.
Pinkerton, M. J. Hein, L. T. Stayner, "Mortality among a Cohort of
Garment Workers Exposed to Formaldehyde: an Update," Occupational and
Environmental Medicine (2004).
[35] Natural Resources Defense Council v. E.P.A., 489 F.3d 1364, 1372-
73 (D.C. Cir, 2007). The court did not specifically address EPA's
reliance on the CIIT study, holding instead that the Clean Air Act
prohibited establishment of the exemptions at issue.
[36] Physiologically based pharmacokinetic models are a class of
dosimetry models that are useful for predicting internal doses to
target organs. With the appropriate data, these models can be used to
extrapolate across species and exposure scenarios and address various
sources of uncertainty in risk assessments.
[37] GAO, Human Health Risk Assessment: EPA Has Taken Steps to
Strengthen Its Process, but Improvements Needed in Planning, Data
Development, and Training, GAO-06-595 (Washington, D.C.: May 31, 2006).
[38] Developing this process, publishing a notice about it in the
Federal Register, and holding a public meeting to obtain public input
on the proposed process has been an item OMB has included in its PART
review of EPA's human health risk assessment program.
[39] Under the proposal, EPA could also identify chemicals "of major
importance" which would be treated as mission-critical chemicals.
[40] Under EPA's draft proposed IRIS assessment process, EPA would
develop, for all chemicals, an additional draft IRIS assessment--one
that includes only qualitative information--that must undergo internal
review. Only draft qualitative assessments for chemicals that are
identified as mission critical would be released for the error
correction step.
[41] According to DOD, the Materials of Emerging Regulatory Interest
Team is a virtual interagency team composed of individuals throughout
DOD with a common interest in emerging contaminants.
[42] Other possible types of DOD risk management investments include
material and process substitutions and personal protective equipment.
[43] The National Academies recently addressed this issue as follows:
"The committee believes that risk assessors and risk managers should
talk with each other; that is, a 'conceptual distinction' does not mean
establishing a wall between risk assessors and risk managers. Indeed
they should have constant interaction. However, the dialogue should not
bias or otherwise color the risk assessment conducted, and the
activities should remain distinct; that is, risk assessors should not
be performing risk management activities." National Academies,
Scientific Review of the Proposed Risk Assessment Bulletin from the
Office of Management and Budget Committee to Review the OMB Risk
Assessment Bulletin (2007).
[44] The National Academies recently emphasized that "stakeholders from
all points on the spectrum of interested parties--other state and
federal agencies, advocacy groups from industry, and affected
communities--can be expected to offer perspectives on the risk
assessment policies under discussion." National Academies, Scientific
Review of the Proposed Risk Assessment Bulletin.
[45] "Other information" includes economic information on the costs and
benefits of mitigating the risk, technological information on the
feasibility of managing the risk, and the concerns of various
stakeholders.
[46] These five assessments address short-term health risks. According
to EPA, while OMB had previously agreed that these assessments would be
included in EPA's 2006 annual performance goals that OMB uses to
evaluate EPA's performance, in November 2007, OMB told EPA that it
would not count EPA's short-term assessments toward meeting its goals,
thereby lowering EPA's performance rating by OMB.
[47] In January 2008, GAO provided OMB staff with the opportunity to
provide comments on the draft report titled "Statement of Facts for
GAO's Review of EPA's Integrated Risk Information System." As you were
aware, from our discussion on January 18, 2008, OMB staff had a number
of concerns about the draft statement of facts, which were discussed in
the January 22, 2008 comments that were sent to you. It appears that
many of those comments have not been incorporated into the current
draft report and our concerns remain. But rather than reiterating them
all here, 1 will highlight our overarching comments on the draft
report.
[48] Draft report, pp. 22-27.
[49] OMB has a broad interest in the quality, objectivity, utility and
integrity of information disseminated by Federal Agencies. See OMB's
Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal Agencies
(2002), 67 Fed. Reg. 8452 (Feb. 22, 2002), available at [hyperlink,
http://www.whitehouse.gov/omb/fedreg/reproducible2.pdf]; OMB's Final
Information Quality Bulletin for Peer Review (2002), 70 Fed. Reg. 2664
(Jan. 14, 2005), available at [hyperlink,
http://www.whitehouse.goviomb/fedreg/2005/011405_peer.pdf]; and the
OMB/OSTP Updated Principles for Risk Analysis, available at [hyperlink,
http://www.whitehouse.gov/omb/memoranda/fy2007/m07-24.pdf].
[50] OMB recognizes that there is also a second round of interagency
review in the current IRIS process. This review seeks to ensure that
comments from expert reviewers and the public have been transparently
and objectively addressed.
[51] Draft report, p. 21.
[52] Draft report, p. 25.
[53] In fact, when the draft report does provide a specific example,
such as its discussion of naphthalene, it does so inaccurately. Draft
report, pp. 36-38. OMB staff does not agree with the characterizations
as presented in the draft report and suggests that readers would
benefit from looking at the EPA documentation.: (i) the EPA charge for
the peer consultation, which resulted from interagency dialogue,
available at [hyperlink,
http://oaspub.epa.gov/eims/eimscomm.getfile?p_download_id=46877]; and
(ii) the peer consultation report, available at [hyperlink,
http://oaspub.epa.gov/eims/eimscomm.getfile?p_download_id=442588].
[54] In fact, the draft report recognizes that it "did not evaluate the
scientific content or quality of IRIS assessments." Draft report, p. 3.
[55] Draft report, p. 26.
[56] We do not consider the interagency process to constitute peer
review; rather the document that EPA releases at a later step in the
process will be subject to external peer review and public comment. EPA
has a very transparent process by which draft IRIS assessments are
released for public comment and external peer review before they are
issued as final documents. The credibility of IRIS assessment is in
part based on this rigorous and transparent external peer review and
public comment process.
[57] Draft report page 6.
[58] See [hyperlink, http://www.iom.edu/CMS/6008.aspx] and [hyperlink,
http://www.nationalacademies.org/studvcommittenrocess.pdf]. In 1997,
Congress ensured that NAS deliberations have heightened confidentiality
protections, when Congress passed stand-alone legislation that added a
new section (Section 15) to the Federal Advisory Committee Act (5
U.S.C. App.) which specifically, and solely, addresses NAS and the
National Academy of Public Administration. See the Federal Advisory
Committee Act Amendments of 1997, Public Law 105-153, 111 Stat. 2689.
The legislative history explains that Congress granted this protection
"to preserve the quality of the research provided to the Federal
Government through the National Academy of Sciences and the National
Academy of Public Administration." Statement of Rep. Horn, Cong. Rec.
1110579 (daily ed. November 9, 1997). Representative Horn explained
that "[t]he administration, the House and the Senate, both the majority
and minority, all agree the academy should not be subject to the full
process of the Federal Advisory Committee Act." Id. As Rep. Horn also
noted, id., Congress passed this legislation in response to the court
decision earlier that year in Animal Legal Defense Fund, Inc. v.
Shalala, 104 F3d 1209 (D.C. Cir.), cert. denied, 522 U.S. 949 (1997),
in which the court held that NAS panels were subject to the full range
of FACA disclosure requirements. In its decision, the court stated that
NAS, in arguing for why it should not be subject to these requirements,
referred to "the deleterious effects of FACA's requirements on its
deliberative processes: As the NAS sees it, open meetings and records
would compromise its internal review procedures and inhibit candid
exchange among its members." Id. at 428.
[59] Draft report, pp. 45-52.
[60] By way of comparison, we note that, in providing to us the draft
report for our comment, your cover page stated that the draft "is
restricted to official use only"; is "subject to revision"; and "must
be safeguarded to prevent improper disclosure."
[61] NLRB v. Sears, Roebuck & Co., 421 U.S. 132, 149 ("it is reasonable
to construe Exemption 5 to exempt those documents, and only those
documents, normally privileged in the civil discovery context"); EPA v.
Mink, 410 U.S. 73, 86 (1973) (The Freedom of Information Act (FOIA)
contemplates that the public is entitled to all memoranda or letters
that a private party could discover in litigation with the agency).
[62] See Chrysler Corp. V. Brown, 441 U.S. 281, 293 (holding that
exemptions from FOIA's general requirement to disclose agency documents
do not prohibit agencies from disclosing information covered by the
exemptions). In fact, OMB has voluntarily released information it
considered to be protected from disclosure under FOIA, presumably
because it saw some benefit in doing so. NRDC v. U.S. Department of
Defense, 442 F.Supp.2d 857, 863 (C.D. Cal. 2006). It is unclear from
the case which FOIA exemptions OMB believed were applicable.
[63] E.g., Mead Data Central, Inc. v. U.S. Dept. of Air Force, 566 F.2d
242, 260 (D.C. Cir. 1977) ("The focus of the FOIA is information, not
documents, and an agency cannot justify withholding an entire document
simply by showing that it contains some exempt material.").
[64] E.g., Bristol-Myers v. FTC, 424 F.2d 935, 939 (FOIA "does not
authorize an agency to throw a protective blanket over all information
by casting it in the form of an internal memorandum. Purely factual
reports and scientific studies cannot be cloaked in secrecy by an
exemption designed to protect only those internal working papers in
which opinions are expressed and policies formulated and recommended");
Southwest Center v. Biological Diversity v. USDA, 170 F.Supp.2d 931,
943 (D. Ariz. 2000) ("FOIA exemption five does not protect research
data"); Verrazzano Trading Corp. v. United States, 349 F.Supp. 1401,
1406 (Cust. Ct. 1972) ("the exemption was not intended to protect
factual or scientific reports and investigations").
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: