Environmental Regulation:

EPA Should Improve Adherence to Guidance for Selected Elements of Regulatory Impact Analyses

GAO-14-519: Published: Jul 18, 2014. Publicly Released: Aug 11, 2014.

Additional Materials:

Contact:

J. Alfredo Gomez
(202) 512-3841
gomezj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Environmental Protection Agency (EPA) used the seven Regulatory Impact Analyses (RIA) GAO reviewed to inform decision making, and its adherence to relevant Office of Management and Budget (OMB) guidance varied. According to senior EPA officials, the agency used these RIAs to facilitate communication with management throughout the rulemaking process and communicate information that supported its regulatory decisions to Congress and the public. However, it generally did not use them as the primary basis for final regulatory decisions.

EPA generally adhered to many aspects of OMB's Circular A-4 guidance for analyzing the economic effects of regulations including, for example, considering regulatory alternatives and analyzing uncertainties underlying its RIAs. However, EPA did not always adhere to other aspects. Specifically, the information EPA included and presented in the RIAs was not always clear. According to OMB guidance, RIAs should communicate information supporting regulatory decisions and enable a third party to understand how the agency arrives at its conclusions. In addition, EPA's review process does not ensure that the information about selected elements that should appear in the analyses—such as descriptions of baselines and alternatives considered—is transparent or clear, within and across its RIAs. As a result, EPA cannot ensure that its RIAs adhere to OMB's guidance to provide the public with a clear understanding of its decision making.

In addition to using Circular A-4 (issued in 2003) to analyze the effects of regulations, EPA used more recent guidance developed by an interagency working group co-led by OMB and another White House office in 2010 for valuing carbon dioxide emissions. Applying this guidance while using Circular A-4 to estimate other benefits and costs yielded inconsistencies in some of EPA's estimates and has raised questions about whether its approach was consistent with Circular A-4. Circular A-4 does not reference the new guidance and the new guidance does not include an overall statement explaining its relationship to Circular A-4. Without increased clarity about the relationship, questions about the agencies' adherence to OMB guidance will likely persist.

In assessing EPA's adherence to OMB guidance, GAO identified two other areas in which EPA faced challenges that limited the usefulness of some of its estimates. First, EPA did not monetize certain benefits and costs related to the primary purposes or key impacts of the rules GAO reviewed, such as reducing hazardous air pollutants and water quality effects. EPA officials said resource and data limitations constrained the agency's ability to monetize these effects. OMB guidance acknowledges that monetizing effects is not always possible. However, without doing so, the public may face challenges understanding the trade-offs associated with regulatory alternatives. Second, EPA estimated effects of its regulations on employment, in part, using a study that, according to EPA officials, represented the best reasonably obtainable data when they conducted their analyses. However, the study was based on data that were more than 20 years old and may not have represented the regulated entities addressed in the RIAs. EPA officials said they are exploring new approaches for analyzing these effects but were uncertain about when such results would be available. Without improvements in its estimates, EPA's RIAs may be limited in their usefulness for helping decision makers and the public understand these important effects.

Why GAO Did This Study

Federal regulations, especially those addressing health, safety, and the environment, can generate hundreds of billions of dollars in benefits and costs to society annually. Various statutes, executive orders, and OMB guidance direct federal agencies to analyze the benefits and costs of proposed regulations. These analyses—known as RIAs—can also provide affected entities, agencies, Congress, and the public with important information about the potential effects of new regulations.

According to OMB, EPA regulations account for the majority of the estimated benefits and costs of major federal regulations. GAO was asked to review EPA's RIAs for recent regulations. This report examines how EPA has used RIAs during the rulemaking process and the extent to which EPA adhered to OMB guidance on selected elements of RIAs for recent rules. GAO reviewed RIAs from a nonprobability sample of seven recent air, water, and other environmental regulations, assessed them against relevant OMB guidance, and interviewed agency officials.

What GAO Recommends

GAO recommends that EPA improve adherence to OMB guidance and enhance the usefulness of its RIAs, and that OMB clarify the application of guidance for estimating the benefits of reducing greenhouse gas emissions. In commenting on a draft of this report, EPA stated that it generally agreed with GAO's recommendations. On behalf of OMB, in oral comments OMB staff said that they neither agreed nor disagreed with the recommendations but saw some merit in them.

For more information, contact J. Alfredo Gomez at (202) 512-3841 or gomezj@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: EPA cited its November 2014 memo that the Office of Policy issued to all program offices regarding transparency and clarity as support that it has begun addressing this recommendation. However, we have no evidence that EPA has taken action to enhance its review process for RIAs to ensure the transparency and clarity of information presented for selected elements in and across RIAs. In March 2016, EPA reiterated that the agency stands behind the quality of RIAs it conducts and does not believe GAO's report identified any systematic deficiencies in the accuracy of our analytical work. EPA also said that consistent with E.O. 12866, the RIAs developed for economically significant regulations are reviewed by the Office of Management and Budget and undergo an interagency review process before being released for public notice and comment, and the agency constantly strives to improve both the quality and transparency of its RIAs. Furthermore, EPA said that its Office of Policy, and particularly the National Center for Environmental Economics (NCEE), has continued to work within the agency's existing Action Development Process to promote transparency and clarity in RIAs. This recommendation remains open because EPA has not enhanced its review process as recommended, and it does not sound like the agency plans to do so.

    Recommendation: To improve future adherence to OMB guidance for conducting RIAs, the EPA Administrator should enhance the agency's review process for RIAs to ensure the transparency and clarity of information presented for selected elements in and across RIAs.

    Agency Affected: Environmental Protection Agency

  2. Status: Closed - Implemented

    Comments: In November 2014, EPA's Office of Policy issued a memo to all program offices requesting that the agency pay particular attention to transparency and clarity as it drafts its RIAs. The memo included references to useful?resources from OMB's and EPA's Economic Guidelines. In addition, the memo noted that the Office of Policy's National Center for Environmental Economics staff are available to assist on improving transparency, clarity, and other aspects of EPA's RIAs. Based on the executive summaries of three regulatory impact assessments EPA has issued since November 2014, GAO believes the agency has taken steps to improve the transparency and clarity of the information included in the executive summaries. For example, in one executive summary, EPA included a detailed two-and-a-half page table identifying the unquantified, quantified, and monetized benefits it considered. In another, EPA included an explicit statement of the baseline, cross-referenced a technical support document where readers could obtain additional detail, and stated what the agency could not monetize. In addition, EPA is providing greater emphasis on transparency and clarity in the economic analysis component of its Action Development Training. Based on these examples and the expectation that the trend toward increased transparency and clarity will continue, GAO is closing this recommendation as implemented.

    Recommendation: To improve future adherence to OMB guidance for conducting RIAs, the EPA Administrator should improve the accuracy, transparency, and clarity of the information included in the executive summaries of each RIA.

    Agency Affected: Environmental Protection Agency

  3. Status: Open

    Comments: In a March 2016 update, OMB reported that the Interagency Working Group continues to monitor the economic literature and will also consider whether there are any new approaches for presenting social cost of carbon estimates with other analytical results that have been discounted at different rates that would help agencies more transparently communicate about the circumstances unique to assessing the long-term effects of changes in carbon dioxide emissions. It added that, as the National Academy of Sciences Committee may address the appropriate use of discount rates in calculating and presenting the social cost of carbon in Phase 2 of its report, it would be prudent to wait for Phase 2 of the report before making changes in this area to ensure that such changes are fully informed by any Committee recommendations. GAO will continue to monitor the Interagency Working Group's progress toward implementing this recommendation.

    Recommendation: To clarify the relationship between OMB Circular A-4 and an Interagency Working Group's Technical Support document for estimating the effects of changes in carbon dioxide emissions, and the approach agencies should use when informing decision makers and the public of their findings, the Director of OMB should consider continuing monitoring the economic literature and working with agencies to identify approaches for presenting social cost of carbon estimates with other analytical results that have been discounted at different rates to help agencies more transparently communicate about the circumstances unique to assessing the long-term effects of changes in carbon dioxide emissions.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  4. Status: Open

    Comments: According to EPA, the agency is working on several critical areas of economic valuation including (1) a long term effort to examine the factors affecting the estimated costs of regulation in a retrospective study of the costs of EPA regulations; (2) convening a Science Advisory Board (SAB) panel (beginning in summer 2015) on modeling economywide impacts; and (3) work underway addressing water quality benefits and non-cancer human health benefits. Regarding (2), the SAB panel met in July and October 2015 and has meetings scheduled for March, May, and July 2016. The SAB panel's charge is, in part, to identify paths forward for improvements in economy-wide models that could address existing limitations and increase their potential utility as analytic tools to support regulatory decisions. EPA is making progress in the spirit of this recommendation. However, GAO is keeping this recommendation open until we can determine how EPA is using the SAB's work to develop its regulatory impact analyses.

    Recommendation: In addition, to enhance the usefulness of EPA's RIAs, the EPA Administrator should identify and prioritize for research key categories of benefits and costs that the agency cannot currently monetize that, once monetized, would most enhance the agency's ability to consider economic trade-offs associated with different regulatory alternatives.

    Agency Affected: Environmental Protection Agency

  5. Status: Open

    Comments: In a March 2016 update, OMB said that in the Interagency Working Group?s next update of the Technical Support Document, the Interagency Working Group will consider further clarification of the relationship between OMB Circular A-4 and the Technical Support Document through increasing the visibility of relevant language in the Technical Support Document.OMB did not specify the time frame for the next update. GAO will continue to monitor the Interagency Working Group's progress toward implementing this recommendation.

    Recommendation: To clarify the relationship between OMB Circular A-4 and an Interagency Working Group's Technical Support document for estimating the effects of changes in carbon dioxide emissions, and the approach agencies should use when informing decision makers and the public of their findings, the Director of OMB should consider clarifying the relationship between OMB Circular A-4 and the Technical Support Document by increasing the visibility of relevant language in the Technical Support Document.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  6. Status: Open

    Comments: EPA told us in October 2015 that their peer review of the employment update to the agency's Guidelines for Economic Analyses is getting underway and the time frame for finalization will depend on the outcome of the peer review process. In March 2016, EPA said it plans to incorporate advice on employment impacts analysis from EPA's Science Advisory Board panel on Economy-Wide Modeling (EWM) and very recent advances in the rapidly developing employment literature into its update on the agency's Guidelines for Preparing Economic Analyses. The EWM SAB panel recently indicated an interest in discussing methods for employment impacts analysis at its upcoming public meeting scheduled for July 19-20, 2016. The SAB panel will address specific charge questions drafted by EPA on methodologies for employment impact analysis. EPA is making progress in the spirit of this recommendation. However, GAO is keeping the recommendation open until seeing the outcome of these efforts in the form of an updated Guidelines for Preparing Economic Analyses.

    Recommendation: In addition, to enhance the usefulness of EPA's RIAs, the EPA Administrator should continue efforts to update and improve the agency's approach to estimating employment effects.

    Agency Affected: Environmental Protection Agency

 

Explore the full database of GAO's Open Recommendations »

Jun 14, 2016

Jun 13, 2016

Jun 2, 2016

Apr 29, 2016

Apr 25, 2016

Mar 28, 2016

Mar 11, 2016

Looking for more? Browse all our products here