EPA Has Started to Address the National Academies' Recommendations on Estimating Health Benefits, but More Progress Is Needed
GAO-06-780, Jul 17, 2006
A large body of scientific evidence links exposure to particulate matter--a widespread form of air pollution--to serious health problems, including asthma and premature death. Under the Clean Air Act, the Environmental Protection Agency (EPA) periodically reviews the appropriate air quality level at which to set national standards to protect the public against the health effects of particulate matter. EPA proposed revisions to these standards in January 2006 and issued a draft regulatory impact analysis of the revisions' expected costs and benefits. The estimated benefits of air pollution regulations have been controversial in the past. A 2002 National Academies report generally supported EPA's approach but made 34 recommendations to improve how EPA implements its approach. GAO was asked to determine whether and how EPA applied the Academies' recommendations in its estimates of the health benefits expected from the January 2006 proposed revisions to the particulate matter standards. GAO examined the draft analysis, met with EPA officials, and interviewed members of the National Academies' committee. In providing technical comments on the report, EPA officials said it was fair and balanced and noted the agency's progress in addressing recommendations via research and development and other analyses.
EPA has begun to change the way it conducts and presents its analyses of health benefits in response to recommendations from the National Academies. Specifically, EPA applied, at least in part, 22--or about two-thirds--of the Academies' recommendations to its health benefit analysis of proposed revisions to particulate matter standards. For example, in response to some of the recommendations, EPA took steps toward conducting a more rigorous assessment of uncertainty by, for instance, evaluating how benefits could change under different assumptions and discussing sources of uncertainty not included in the benefit estimates. In one case, EPA applied an alternative technique, called expert elicitation, for evaluating uncertainty by systematically gathering expert opinion about the uncertainty underlying the causal link between exposure to particulate matter and premature death. Consistent with the National Academies' recommendation to assess uncertainty by developing ranges of estimates and specifying the likelihood of attaining them, EPA used expert elicitation to develop ranges of reductions in premature death expected from the proposed revisions. EPA officials said that ongoing research and development efforts will allow the agency to gradually achieve more progress in applying the recommendations. We note that robust uncertainty analysis is important because estimates of health benefits can be highly uncertain, as the draft regulatory impact analysis for particulate matter illustrates. EPA viewed the estimates in this analysis as so uncertain that it chose not to present them in the executive summary. For various reasons, EPA has not applied the remaining 12 recommendations to the analysis, such as the recommendation to evaluate the impact of using the simplifying assumption that each component of particulate matter is equally toxic. EPA officials viewed most of these recommendations as relevant to its health benefit analyses and, citing the need for additional research and development, emphasized the agency's commitment to continue to respond to the recommendations. For example, EPA did not believe that the state of scientific knowledge on the relative toxicity of particulate matter components was sufficiently developed to include in the January 2006 regulatory impact analysis, and the agency is currently sponsoring research on this issue. In addition, a senior EPA official said that insufficient resources impeded the agency's progress in applying the recommendations, citing, in particular, the limited availability of skilled staff, time, and other resources to conduct the required analyses and research and development. EPA officials also said that some of the recommendations the agency did not apply to the draft analysis, such as one calling for a summary table describing key analytical information to enhance transparency, will be applied to the analysis supporting the final rule. To the extent that EPA continues to make progress addressing the Academies' recommendations, decision makers and the public will be better able to evaluate the basis for EPA's air regulations.