This is the accessible text file for GAO report number GAO-14-268 
entitled 'Reexamining Regulations: Agencies Often Made Regulatory 
Changes, but Could Strengthen Linkages to Performance Goals' which was 
released on May 12, 2014. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

April 2014: 

Reexamining Regulations: 

Agencies Often Made Regulatory Changes, but Could Strengthen Linkages 
to Performance Goals: 

GAO-14-268: 

GAO Highlights: 

Highlights of GAO-14-268, a report to congressional requesters. 

Why GAO Did This Study: 

Federal agencies issue thousands of regulations annually to address 
such national goals as public health and safety. Retrospective 
analysis can help agencies evaluate how existing regulations work in 
practice. GAO was asked to provide insights on agencies' retrospective 
analyses. This report identifies for selected agencies (1) the results 
and anticipated outcomes of retrospective analyses agencies completed, 
(2) strategies, practices, or factors that affected agencies' ability 
to implement these analyses, and (3) the extent to which agencies are 
incorporating the analyses into processes for measuring and achieving 
agency priority goals. Applying criteria from executive orders, 
GPRAMA, and related guidance, GAO analyzed documents from 22 executive 
agencies and 2 independent regulatory agencies that prepared final 
retrospective review plans. These agencies issued more than 96 percent 
of all final rules published between 2011 and 2013. GAO also obtained 
agency officials' views through questionnaires and a roundtable of 9 
agencies selected primarily on numbers of completed analyses. The 
officials' views are not generalizable to all agencies. 

What GAO Found: 

Agencies often made changes to regulations in response to completed 
retrospective regulatory analyses, but could improve the reporting of 
progress. Executive Orders and related implementation guidance from 
the Office of Management and Budget (OMB) require executive agencies, 
and encourage independent regulatory agencies, to develop and 
implement retrospective review plans. Agencies use semiannual updates 
to report on the progress and results of their analyses. The 22 
executive agencies in GAO's scope identified more than 650 planned 
analyses and reported having completed and taken final actions on 246 
of those analyses by August 31, 2013. The two independent regulatory 
agencies in GAO's scope each chose to develop a final retrospective 
review plan, although not required to do so. More than 90 percent of 
the completed analyses led executive agencies to revise, clarify, or 
eliminate regulatory text. Agencies also took other actions such as 
updating guidance to the public. Agencies most commonly reported three 
expected outcomes from actions they took: improving the effectiveness 
of regulations (112 of 246), reducing regulatory burden (99 of 246), 
and clarifying regulations or making other administrative changes (93 
of 246), such as implementing new procedures. Agencies often reported 
more than one outcome. Agencies quantified expected savings for 38 of 
the 246 completed analyses, often attributing savings to reduced 
information collection burdens. However, agencies did not consistently 
include citations or links to the supporting analyses and data in 
their progress reports. While OMB guidance contains transparency 
requirements for agencies to inform the public, OMB could work with 
agencies to effectively implement the guidance to improve the 
usefulness of the information agencies report on the results of their 
analyses. 

Officials from the 9 agencies that participated in GAO's roundtable 
identified three key strategies and two barriers that most often 
affected their implementation of retrospective analyses. Strategies 
that facilitated planning and conducting analyses included: (1) 
establishing a centrally coordinated review process, (2) leveraging 
existing regulatory activities such as rulemaking and enforcement 
processes, and (3) using existing external feedback mechanisms such as 
advisory committees. The most commonly cited barriers were competing 
priorities for available staff and difficulty obtaining sufficient 
data. 

Retrospective analysis can also help inform agencies' priority goals 
(APG). The Government Performance and Results Act Modernization Act of 
2010 (GPRAMA) requires agencies to assess whether regulations, among 
other activities, are contributing as planned to APGs. Agencies 
reported mixed experiences linking retrospective analyses to APGs. The 
seven roundtable agencies with APGs identified regulations 
contributing to their priority goals, but their retrospective analyses 
were only sometimes linked to APGs. In some cases, different offices 
in the same agency had mixed responses about whether such linkages 
existed. Several agency officials said staff conducting retrospective 
analyses were not involved in performance discussions at higher levels 
of the agency. To inform broader performance planning and reviews, 
retrospective analyses can be another potential data source for APGs. 
Agencies could strengthen that linkage by taking actions such as 
considering APGs, to the extent practicable, when planning 
retrospective analyses and identifying how they will measure the 
performance of significant new rules related to priority goals. 

What GAO Recommends: 

GAO recommends that OMB work with agencies to improve reporting on 
results of retrospective analyses and strengthen links between those 
analyses and agencies' performance goals by considering APGs when 
planning retrospective analyses, among other actions. OMB staff 
generally agreed with the recommendations in this report. 

View [hyperlink, http://www.gao.gov/products/GAO-14-268]. For more 
information, contact Michelle Sager at (202) 512-6806 or 
sagerm@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Almost All Completed Retrospective Analyses Resulted in Changes to 
Regulations or Other Actions, but Progress Updates Could Be Improved: 

Agency Officials Identified Strategies and Barriers That Affected 
Their Implementation of Retrospective Analyses: 

Opportunities Exist to Improve Linkages between Regulatory Reviews and 
Priority Goals: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Key Retrospective Analysis Requirements Listed in 
Executive Orders and Memorandums Issued Since January 2011: 

Appendix II: Staff Contact and Acknowledgments: 

Tables: 

Table 1: Number of Planned and Completed Retrospective Regulatory 
Analyses for 22 Executive Agencies between January 2011 and August 
2013: 

Table 2: Executive Orders and Memorandums Issued Since January 2011 
Related to Retrospective Regulatory Analysis: 

Figures: 

Figure 1: Overview of Retrospective Analysis Results for Executive 
Agencies That Implemented Final Actions between January 2011 and 
August 2013: 

Figure 2: Breakdown of the Types of Reported Retrospective Analysis 
Outcomes for Executive Agencies That Implemented Final Actions from 
January 2011 through August 2013: 

Figure 3: DOT Example of How Regulations Link to an Agency Performance 
Goal: 

Abbreviations: 

APG: agency priority goal: 

ATF: Bureau of Alcohol, Tobacco, Firearms, and Explosives: 

BIS: Bureau of Industry and Security: 

CBP: U.S. Customs and Border Protection: 

CFO Act: Chief Financial Officers Act: 

CFRC: ode of Federal Regulations: 

CFTC: Commodity Futures Trading Commission: 

Commerce: Department of Commerce: 

DHS: Department of Homeland Security: 

DOE: Department of Energy: 

DOJ: Department of Justice: 

DOT: Department of Transportation: 

EHR: Electronic Health Record: 

EPA: Environmental Protection Agency: 

FCC: Federal Communications Commission: 

FTC: Federal Trade Commission: 

GPRA: Government Performance and Results Act of 1993: 

GPRAMA: GPRA Modernization Act of 2010: 

HHS: Department of Health and Human Services: 

OIRA: Office of Information and Regulatory Affairs: 

OMB: Office of Management and Budget: 

PIC: Performance Improvement Council: 

State: Department of State: 

Unified Agenda: Unified Agenda of Federal Regulatory and Deregulatory 
Actions: 

[End of section] 

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548: 

April 11, 2014: 

The Honorable Ron Johnson: 
Ranking Member: 
Subcommittee on Financial and Contracting Oversight: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Mark R. Warner: 
Chairman: 
Task Force on Government Performance: 
Committee on the Budget: 
United States Senate: 

Regulation is one of many tools of government that federal agencies 
use to achieve such national goals as protecting the health and safety 
of the public and the environment.[Footnote 1] Each year, agencies 
issue thousands of rules, within the scope of their authorizing 
statutes, to address these goals. In light of their substantial social 
and economic benefits and costs, agencies must comply with multiple 
procedural and analytical requirements before issuing significant 
regulations.[Footnote 2] However, without careful oversight of those 
regulations once implemented, they might prove to be less effective 
than expected in achieving their intended goals, become outdated, or 
create unnecessary burdens. Regulations may also change the behaviors 
of regulated entities and the public in ways that cannot be predicted 
by prospective analysis before implementation. For these reasons, 
retrospective analysis can be important in evaluating how existing 
regulations are working in practice. Agencies could use retrospective 
analysis to examine how existing regulations have contributed to 
specific policy goals, to assess the effectiveness of their 
implementation, or to reexamine their estimated benefits and costs 
based on actual performance and experience. 

For many years Congress and presidents have directed agencies to 
periodically review existing regulations through various statutes and 
executive orders. In addition, agencies often exercise their own 
discretionary authorities to review existing regulations, as we 
reported in 2007.[Footnote 3] To further promote and institutionalize 
retrospective regulatory analysis, among other things, in 2011 and 
2012 the administration issued Executive Orders 13563, 13579, and 
13610, along with related Office of Management and Budget (OMB) 
guidance.[Footnote 4] Among other provisions, those executive orders 
and guidance documents require executive agencies--and encourage 
independent regulatory agencies--to develop and implement 
retrospective review plans.[Footnote 5] Subsequent updates of 
agencies' plans (now produced semiannually by executive agencies) also 
serve as the primary vehicle for agencies to report on the progress 
and results of their retrospective analyses. Furthermore, as a 
separate, broader effort to improve the performance of government 
agencies and provide greater accountability for results, Congress 
updated the statutory framework for performance management in the 
federal government--the Government Performance and Results Act of 1993 
(GPRA)--with the GPRA Modernization Act of 2010 (GPRAMA).[Footnote 6] 
Particularly relevant to this report, GPRAMA requires agencies to 
assess whether regulations, among many other tools and activities, are 
contributing as planned to agency priority goals, and to review their 
progress on those goals quarterly. 

In this context, you asked us to provide insights concerning the 
scope, effectiveness, and results of agencies' retrospective 
regulatory analyses. This report identifies for selected agencies (1) 
the results and anticipated outcomes of completed retrospective 
analyses included in agencies' review plans and progress reports, (2) 
strategies, practices, or factors that facilitated or limited 
agencies' ability to implement retrospective analyses, and (3) the 
extent to which agencies are incorporating retrospective analyses into 
their processes for measuring and achieving agency priority goals. 

To address our first objective, we reviewed relevant policies, 
executive orders, and related guidance to identify criteria. We 
analyzed published final retrospective regulatory review plans and 
progress reports from 22 executive agencies and 2 independent 
regulatory agencies that published final and updated retrospective 
analyses between January 2011 and August 2013.[Footnote 7] In 
addition, we analyzed in greater detail those completed analyses that 
resulted in agencies implementing a final action by August 31, 2013. 
These 24 agencies issued more than 96 percent of all final rules 
published within the time period covered in our review. The agencies 
within our scope include the Departments of Agriculture, Commerce, 
Defense, Education, Energy (DOE), Health and Human Services (HHS), 
Homeland Security (DHS), Housing and Urban Development, Interior, 
Justice, Labor, State, Transportation (DOT), Treasury, and Veterans 
Affairs. We also included the Environmental Protection Agency (EPA), 
General Services Administration, National Aeronautics and Space 
Administration, Office of Personnel Management, Pension Benefit 
Guaranty Corporation, Small Business Administration, Social Security 
Administration, and, from the independent regulatory agencies, the 
Federal Communications Commission (FCC) and Federal Trade Commission 
(FTC). Using criteria from Executive Orders 13563, 13579, and 13610, 
and related guidance, we assessed information that agencies reported 
on concerning the implementation of their plans. This included the 
number of completed retrospective analyses, the results and outcomes 
agencies reported, and whether agencies addressed other elements 
identified by the guidance. We specifically focused on those 
retrospective analyses that agencies had completed and for which they 
had taken a final action (including a decision to take no further 
action).[Footnote 8] We based our analysis of the results on the 
descriptions agencies reported in their retrospective review plans and 
updates, including anticipated outcomes agencies expect to realize in 
future years. Thus, we did not assess the accuracy or achievement of 
agencies' reported outcomes. 

To test the reliability of the data sources we used to identify 
agencies' final and updated retrospective analyses, we corroborated 
key elements from the agencies' plans and updates, such as the 
reported completion dates and expected outcomes of individual 
analyses. We did this by reviewing related published documents such as 
Federal Register notices and the Unified Agenda of Federal Regulatory 
and Deregulatory Actions (Unified Agenda) and the Regulatory Plan. 
[Footnote 9] We also interviewed officials within OMB's Office of 
Information and Regulatory Affairs (OIRA) and confirmed that we 
included in our scope the most up-to-date agency review plans. We 
concluded that the data were sufficiently reliable for the purpose of 
identifying the scope and results of agencies' retrospective analyses. 

To address our second and third objectives, we selected nine agencies 
to participate in a structured questionnaire about their 
implementation of retrospective reviews, barriers and facilitators in 
using retrospective analyses to make regulatory decisions, and 
incorporating analyses into agency performance management.[Footnote 
10] Subsequently, representatives from the nine agencies participated 
in a more detailed roundtable discussion. Our selection process was 
based primarily on those agencies that completed the greatest number 
of retrospective analyses during the time period covered in our 
review. In addition, we selected agencies whose analyses reported a 
mix of anticipated outcomes (such as reduced burden) and quantified 
costs and benefits. The selected agencies also reflect a mix of 
executive and independent regulatory agencies' experiences. The nine 
agencies are: the Departments of Commerce, Energy, Health and Human 
Services, Homeland Security, State, and Transportation, the 
Environmental Protection Agency, the Federal Communications 
Commission, and the Federal Trade Commission. The views of officials 
from these agencies are not generalizable to all agencies, but 
provided insights from those agencies that most often met our 
selection criteria. We also met with OMB staff to supplement 
information obtained from the questionnaires and roundtable. 

We conducted this performance audit from May 2013 to April 2014 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

Retrospective Regulatory Analysis: 

Regulations are the means by which agencies establish legally binding 
requirements, codified in the Code of Federal Regulations (CFR). 
[Footnote 11] Typically, regulations require a desired action or 
prohibit certain actions by regulated parties. The regulatory process 
is governed by statutes, executive orders, and agencies' policies and 
procedures that require agencies to evaluate the need for regulations, 
assess the potential effects of new regulations, and obtain public 
input (with certain exceptions) during their development. In addition, 
under various congressional and presidential directives, agencies are 
expected to systematically review regulations to ensure they do not 
become outdated or overly burdensome, among other things.[Footnote 12] 
Agencies also may choose to review regulations at their own 
discretion, in response to feedback from regulated parties and agency 
staff or changes in technology and market conditions, among other 
inputs. Therefore, the normal rulemaking process often involves 
agencies reviewing their existing regulations.[Footnote 13] Over many 
years, we have reported on agencies' activities related to 
retrospective regulatory analysis and have demonstrated its usefulness 
to Congress, agencies, and the public while recognizing the potential 
difficulties, such as data limitations and overlapping or duplicative 
requirements. 

In 2011 and 2012, the administration issued new directives to agencies 
on how they should plan and assess analyses of existing regulations, 
among other subjects. 

* Executive Order 13563, "Improving Regulation and Regulatory Review," 
and related memorandums, directed each executive agency to develop and 
submit to OIRA a preliminary plan consistent with law and its 
resources and regulatory priorities under which the agency will 
periodically review its existing significant regulations to determine 
whether they should be modified, streamlined, expanded, or repealed, 
so as to make the agency's regulatory program more effective or less 
burdensome in achieving the regulatory objectives.[Footnote 14] 

* Executive Order 13579, "Regulation and Independent Regulatory 
Agencies," encouraged independent regulatory agencies to develop and 
release to the public plans for retrospective analyses of existing 
significant rules.[Footnote 15] 

* Executive Order 13610, "Identifying and Reducing Regulatory 
Burdens," contained additional requirements and guidance regarding 
public participation in retrospective reviews, setting priorities in 
implementing retrospective review plans (for example, focusing on 
initiatives that would reduce regulatory burdens), and agencies' 
reporting on the status of their retrospective review efforts. 
[Footnote 16] 

OMB periodically issues guidance to agencies on implementing executive 
orders. Under Executive Order 12866, OMB's OIRA is responsible for 
providing meaningful guidance and oversight so that each agency's 
regulations are consistent with applicable law, the President's 
priorities, and the principles set forth in executive orders. For 
example, on October 26, 2011, the Administrator of OIRA issued a 
memorandum on the implementation of the retrospective review plans 
called for by Executive Order 13563. The memorandum provided more 
detailed guidance to agencies on how to address elements of the order 
on prioritizing review plans, seeking public comment, and reporting on 
the status of retrospective review efforts (including a suggested 
template for agencies to use for their reports). According to 
Executive Order 12866, OIRA is to be the repository of expertise 
concerning regulatory issues. 

As we reported in 2007, agencies generally assess their existing 
regulations for a variety of purposes, including to determine whether 
(1) the expected outcomes of the regulation have been achieved, (2) 
the agency should retain, amend, or rescind the regulation, or (3) the 
actual benefits and costs of the implemented regulation correspond 
with estimates prepared at the time the regulation was issued. 
However, OMB's guidance on the 2011 and 2012 orders particularly 
advised agencies to identify in their final plans specific reforms and 
initiatives that will significantly reduce existing regulatory burdens 
(including paperwork and reporting burdens) and promote economic 
growth and job creation. OMB also directed agencies to provide 
specific timelines and deadlines for implementation and to quantify 
burden and cost reductions to the extent feasible. Among other 
components of guidance on the conduct of retrospective analyses, 
agencies are expected to: 

* conduct a quantifiable assessment of the current costs and benefits 
of any proposed changes to existing regulations to the extent possible, 

* develop plans for how they will measure the performance of 
regulations in the future, 

* seek the public's views on retrospective review plans and related 
analysis, and: 

* coordinate the development of the new retrospective plans with other 
retrospective review requirements. 

The executive orders and related guidance addressed all seven of the 
recommendations we made in our 2007 report to ensure that agencies 
conduct effective and transparent retrospective reviews. Among the 
elements that we recommended incorporating in policies, procedures, or 
guidance were: minimum standards for documenting and reporting 
completed review results; including public input as a factor in 
regulatory review decisions; and consideration of how agencies will 
measure the performance of new regulations. For more information on 
key elements of the executive orders and guidance, which we used as 
criteria during our review, see appendix I. 

Agency Priority Goals: 

GPRAMA requires the 24 agencies identified in the Chief Financial 
Officers (CFO) Act, or as otherwise determined by OMB, to develop 
agency priority goals (APG) every 2 years.[Footnote 17] These goals, 
which are a subset of the agency's performance goals, are to reflect 
the highest priorities of each agency, as identified by the head of 
the agency, and be informed by broad crosscutting federal government 
priority goals as well as input from relevant congressional 
committees. Agencies are to identify the various regulations, as well 
as federal organizations, program activities, policies and other 
activities (both within and external to the agency) that contribute to 
each of their APGs and review progress quarterly. This information is 
to be included in agencies' performance plans which are submitted to 
OMB for publication on Performance.gov.[Footnote 18] In April 2013, we 
reported that OMB's review process does not systematically check 
whether agencies have identified all relevant contributors to APGs. 
[Footnote 19] In that report we therefore recommended that OMB work to 
ensure that agencies adhere to its guidance for website updates by 
providing complete information about regulations, among other things, 
that contributed to each APG. OMB staff agreed with this 
recommendation.[Footnote 20] 

Almost All Completed Retrospective Analyses Resulted in Changes to 
Regulations or Other Actions, but Progress Updates Could Be Improved: 

The 22 executive agencies within the scope of our review identified 
665 initiatives (planned retrospective regulatory analyses) in the 
final review plans and subsequent updates prepared in response to 
Executive Orders 13563 and 13610. Between January 2011 and the end of 
August 2013, the agencies completed and had taken at least some final 
action on 246 of those planned analyses (see table 1).[Footnote 21] 
Because agencies' analyses are part of an ongoing process, the 
information in table 1 represents a snapshot of agencies' progress 
reported as of August 31, 2013. 

Table 1: Number of Planned and Completed Retrospective Regulatory 
Analyses for 22 Executive Agencies between January 2011 and August 
2013: 

Agency: Department of Transportation; 
Planned retrospective analyses: 98; 
Completed analyses with final action implemented by August 31, 2013: 
36. 

Agency: Department of Health and Human Services; 
Planned retrospective analyses: 83; 
Completed analyses with final action implemented by August 31, 2013: 
33. 

Agency: Department of State; 
Planned retrospective analyses: 73; 
Completed analyses with final action implemented by August 31, 2013: 
31. 

Agency: Department of Commerce; 
Planned retrospective analyses: 46; 
Completed analyses with final action implemented by August 31, 2013: 
24. 

Agency: Environmental Protection Agency; 
Planned retrospective analyses: 37; 
Completed analyses with final action implemented by August 31, 2013: 
18. 

Agency: Small Business Administration; 
Planned retrospective analyses: 27; 
Completed analyses with final action implemented by August 31, 2013: 
13. 

Agency: Department of Defense; 
Planned retrospective analyses: 37; 
Completed analyses with final action implemented by August 31, 2013: 
11. 

Agency: Department of Energy; 
Planned retrospective analyses: 18; 
Completed analyses with final action implemented by August 31, 2013: 
11. 

Agency: National Aeronautics and Space Administration; 
Planned retrospective analyses: 27; 
Completed analyses with final action implemented by August 31, 2013: 
11. 

Agency: Department of the Treasury; 
Planned retrospective analyses: 18; 
Completed analyses with final action implemented by August 31, 2013: 9. 

Agency: Department of Homeland Security; 
Planned retrospective analyses: 21; 
Completed analyses with final action implemented by August 31, 2013: 8. 

Agency: Department of Housing and Urban Development; 
Planned retrospective analyses: 17; 
Completed analyses with final action implemented by August 31, 2013: 7. 

Agency: Office of Personnel Management; 
Planned retrospective analyses: 16; 
Completed analyses with final action implemented by August 31, 2013: 7. 

Agency: Department of Labor; 
Planned retrospective analyses: 20; 
Completed analyses with final action implemented by August 31, 2013: 7. 

Agency: Department of Education; 
Planned retrospective analyses: 15; 
Completed analyses with final action implemented by August 31, 2013: 6. 

Agency: Department of the Interior; 
Planned retrospective analyses: 16; 
Completed analyses with final action implemented by August 31, 2013: 4. 

Agency: General Services Administration; 
Planned retrospective analyses: 34; 
Completed analyses with final action implemented by August 31, 2013: 4. 

Agency: Pension Benefit Guaranty Corporation; 
Planned retrospective analyses: 14; 
Completed analyses with final action implemented by August 31, 2013: 3. 

Agency: Social Security Administration; 
Planned retrospective analyses: 16; 
Completed analyses with final action implemented by August 31, 2013: 3. 

Agency: Department of Agriculture; 
Planned retrospective analyses: 13; 
Completed analyses with final action implemented by August 31, 2013: 0. 

Agency: Department of Justice; 
Planned retrospective analyses: 2; 
Completed analyses with final action implemented by August 31, 2013: 0. 

Agency: Department of Veterans Affairs; 
Planned retrospective analyses: 17; 
Completed analyses with final action implemented by August 31, 2013: 0. 

Agency: Total; 
Planned retrospective analyses: 665; 
Completed analyses with final action implemented by August 31, 2013: 
246. 

Source: GAO analysis of regulatory agencies' retrospective regulatory 
review plans and updates. 

Note: For the purpose of this review, we counted the number of 
retrospective analyses rather than the number of regulations that may 
have been included in an analysis. A single analysis can cover more 
than one regulation and the number of regulations covered in an 
analysis can vary. 

[End of table] 

Executive Agencies Often Changed Regulations to Improve Effectiveness 
or Reduce Burden: 

Agencies' reports on the progress of their planned retrospective 
analyses and related documentation showed that analyses completed by 
the end of August 2013 almost always resulted in the agencies taking 
additional specific actions, as shown in figure 1. 

Figure 1: Overview of Retrospective Analysis Results for Executive 
Agencies That Implemented Final Actions between January 2011 and 
August 2013: 

[Refer to PDF for image: illustration] 

Agency completed retrospective analyses: 246. 

Agency determined no additional action needed: 3. 

Agency determined additional action needed: 243: 
* Agency amended the Code of Federal Regulations: 225; 
* Agency took other type of action: 18. 

Source: GAO analysis of regulatory agencies' retrospective regulatory 
review plans and updates. 

Note: For purposes of this report, we defined a completed 
retrospective analysis as one for which the agency both completed the 
analysis and implemented a final action by August 31, 2013. A final 
action includes instances where an agency determined no additional 
action was needed. Although the two subcategories under "additional 
action needed" were mutually exclusive in the review conducted for 
this report, this may not always be the case. A completed analysis 
could lead an agency to both amend the CFR and take other actions 
affecting the reviewed regulation. 

[End of figure] 

In 225 of the 246 completed analyses we examined (more than 90 
percent), the reviews led to agencies amending sections of the CFR to 
revise, clarify, or eliminate regulatory text. For example, DHS 
amended regulations administered by U.S. Customs and Border Protection 
(CBP) to allow for the publication of seizure and intent-to-forfeit 
notices on an official government forfeiture website.[Footnote 22] 
Previously, notices were posted for three successive weeks in a 
newspaper circulated at the CBP port and in the judicial district 
where CBP seized the property. According to DHS, the changes are 
intended to reduce administrative costs and improve the effectiveness 
of CBP's notice procedures as Internet publication will reach a 
broader range of the public and provide access to more parties who may 
have an interest in the seized property. 

Another 18 of the 246 completed retrospective analyses resulted in 
agencies taking some other type of action, such as revising agency 
guidance or evaluation criteria, affecting implementation of the 
regulations that had been reviewed. For example, DOT conducted a 
public meeting in February 2012 concerning the criteria used when 
determining an applicant's minimum level of fitness for special 
permits and approvals by the Pipeline and Hazardous Materials Safety 
Administration under hazardous materials regulations. Based on the 
comments DOT received during the meeting, the agency decided to 
undertake a major restructuring of the fitness determinations process 
and may consider rulemaking once the restructuring is complete. The 
remaining three completed analyses within our scope resulted in an 
agency determining that no further action was needed. For example, in 
one of its completed analyses, EPA sought public feedback on a 2003 
water quality trading policy to determine whether revisions to the 
related policy could help increase the adoption of market-based 
approaches. But after holding public workshops and webinars on the 
subject, the agency decided that no revisions were necessary.[Footnote 
23] 

Agencies most commonly identified three categories of expected 
outcomes from the actions they took in response to completed 
retrospective analyses: (1) improving the effectiveness of 
regulations; (2) reducing the burden on regulated entities; and (3) 
providing clarity on regulations or making other administrative 
changes. In addition, agencies also amended regulations to address 
statutory changes. These outcomes are not mutually exclusive because 
agencies reported multiple categories of outcomes for many of the 
completed analyses. Figure 2 displays a breakdown of the expected 
outcomes that agencies reported for their completed analyses. 

Figure 2: Breakdown of the Types of Reported Retrospective Analysis 
Outcomes for Executive Agencies That Implemented Final Actions from 
January 2011 through August 2013: 

[Refer to PDF for image: horizontal bar graph] 

Types of reported retrospective analysis outcomes: 

Agency improved effectiveness of the regulation: 112; 
Agency reduced burden of the regulation: 99; 
Agency provided clarity or made another administrative change: 93; 
Agency amended regulation to respond to statutory change: 44. 

Source: GAO analysis of regulatory agencies' retrospective regulatory 
review plans and updates. 

Notes: Results represent 19 agencies and 246 completed retrospective 
analyses. The outcome categories are not mutually exclusive. Agencies 
reported outcomes in multiple categories for 76 of the completed 
analyses. 

[End of figure] 

* Agencies identified improving the effectiveness of regulations as an 
expected outcome of 112 completed retrospective analyses. For example, 
DOT revised its regulations related to Railroad Locomotive Safety 
Standards in 2012 to improve their effectiveness by incorporating 
current industry and engineering best practices related to locomotive 
electronics.[Footnote 24] The number of outcomes in this category 
underscores one of the findings in our 2007 report, when agencies said 
that their primary purpose for conducting most retrospective analyses 
is to examine the effectiveness of the implementation of regulations. 
[Footnote 25] 

* Agencies cited reducing burden on regulated parties as an 
anticipated outcome of 99 completed retrospective analyses. For 
example, in October 2012 the Department of Commerce's (Commerce) 
National Oceanic and Atmospheric Administration removed income-
qualification requirements related to renewing Gulf of Mexico 
commercial reef fish permits and increased the maximum crew size for 
vessels that possess both charter and commercial permits.[Footnote 26] 
The stated intent of this rule is to remove permit requirements the 
agency viewed as no longer applicable to current commercial fishing 
practices and to improve safety at sea. The prevalence of outcomes in 
this category is consistent with one of the central goals of the 
executive orders and OMB guidance on the retrospective analysis 
initiative--reducing unnecessary regulatory burdens and costs on 
individuals, businesses, and state, local, and tribal governments. 

* For 93 of the completed analyses, agencies indicated that the 
actions they took would clarify regulations or make another type of 
administrative change. For example, in response to one of its 
analyses, the Department of State (State) implemented new procedures 
for its Summer Work Travel Program in 2011. The agency clarified which 
U.S. host employers and third-party overseas agents or partners could 
assist in the administration of the core functions of exchange 
programs.[Footnote 27] 

* We found 44 instances of agencies amending existing regulations in 
response to statutory changes. For example, DOE published a final rule 
in July 2013 to implement provisions in the Energy Conservation and 
Production Act that require the department to update the baseline 
federal energy efficiency performance standards for the construction 
of new federal commercial and multi-family high-rise residential 
buildings.[Footnote 28] 

Executive Agencies Largely Attributed Quantified Savings to Reducing 
Burden: 

Executive Order 13610 directed agencies to give priority to regulatory 
reforms that would produce significant quantifiable monetary savings 
or reductions in paperwork burdens, while protecting public health, 
welfare, safety, and the environment.[Footnote 29] OMB's guidance 
further directed agencies to quantify burden and cost reductions to 
the extent feasible when reporting on the progress of their 
retrospective analyses.[Footnote 30] Agencies identified quantified 
cost savings in the published progress updates regarding 38 of the 246 
completed analyses in our scope. Half of those 38 analyses focused on 
reducing burden, including reducing the costs of information 
collection burdens.[Footnote 31] For example, HHS issued final rules 
in May 2012 to remove unnecessary regulatory and reporting 
requirements for hospitals and other health care providers.[Footnote 
32] The agency reduced the burden to providers and suppliers by 
modifying, removing, or streamlining regulations that the agency 
identified as excessively burdensome. For example, HHS eliminated a 
requirement for non-physician personnel to have special training in 
administering blood transfusions and intravenous medications. The 
agency expects these changes to yield more than $5 billion in savings 
over the next 5 years. In the 38 cases where agencies quantified 
savings estimates, the extent to which agencies reported information 
about the methodologies, assumptions, and time periods used to develop 
those estimates varied. For example, not all agencies identified in 
their progress reports the discount rate used to discount future 
savings. In addition, agencies sometimes reported savings estimates 
based on varying time frames. Without additional information on 
agencies' methodologies and key assumptions used to estimate savings 
in their plan updates and progress reports, a reader cannot determine 
the total cumulative amount of savings that agencies expected to 
result from these retrospective analyses. Including more detailed 
information behind savings estimates would also be consistent with the 
internal control standard for information and communication, which 
states that agencies must have relevant, reliable, and timely 
information for both external and internal uses.[Footnote 33] Such 
information could also help Congress and the public to better 
understand the basis for agencies' estimates and the extent to which 
estimates may be comparable. 

In other cases, agencies identified but did not quantify expected 
savings. For example, HHS encouraged that all clinical quality 
measures be electronically reported beginning in 2014 as part of the 
Medicare and Medicaid Electronic Health Record (EHR) Incentive 
Program.[Footnote 34] Instead of entering data manually or assembling 
paperwork, providers would instead generate a file from the EHR 
system. HHS reported that it expects this change to result in savings, 
but the agency did not quantify such savings. Agency officials we 
spoke with described impediments to quantifying the costs and benefits 
of their regulations, such as obtaining sufficient data, which we will 
discuss further in a later section of this report. 

Executive Agencies Increased the Transparency of Review Plans, but 
Could Improve the Usefulness of Progress Reports to the Public: 

The executive orders and related OMB guidance contain transparency 
requirements intended to better inform the public and solicit input 
about agencies' retrospective analyses. The orders outline general 
principles, along with some basic requirements, while guidance 
memorandums provide more detailed explanations and instructions on how 
agencies should implement the orders. Among these requirements, 
agencies were directed to make their retrospective analyses available 
to the public within a reasonable time period and to release their 
analyses, including the supporting data, online whenever possible. OMB 
guidance further recommended that agencies post their analyses on 
their individual "Open Government" web pages.[Footnote 35] 

Consistent with executive orders and OMB guidance, almost all of the 
agencies included in our review published retrospective review plans 
either on their agency websites or other locations. Also, 20 published 
drafts of their plans in the Federal Register for public comment. 
Overall, this represents greater transparency compared to conditions 
we found in our 2007 report. In 2007, we found limitations in 
agencies' documentation and reporting of discretionary retrospective 
analyses. In addition, nonfederal parties cited this lack of 
transparency as a barrier to the usefulness of retrospective analyses 
to the public. 

Although many agencies in our scope for this review posted their 
retrospective review plans online, the ease of finding review plans 
and the comprehensiveness of the plans varied across agencies. For 
example, agencies did not always post plans on their "Open Government" 
web pages as recommended in OMB guidance. In some cases, agencies 
directed readers to multiple plans to identify outcomes from past 
retrospective analyses, but did not always explicitly state whether 
the current plan included or excluded a comprehensive list of analyses 
and outcomes. The White House web page, which posts a compiled list of 
agency plans, did not always include agencies' semiannual progress 
reports showing the outcomes and progress of analyses. The last 
updates posted on that web page were in May 2012.[Footnote 36] OMB 
OIRA staff provided us with updated information in April 2014 
explaining that the agency made a deliberate decision to discontinue 
updating the list of progress updates on the White House web page in 
an effort to reinforce that the responsibility reside with the 
individual agencies to post all of their plans and updates on their 
"Open Government" web pages.[Footnote 37] During our review, agencies' 
plan updates and progress reports provided only summary information 
about completed analyses. Agencies did not always provide citations or 
references in the progress reports that a reader could use to look up 
published rules that contain more detailed descriptions of agencies' 
analyses and the underlying data. Without more consistent and 
comprehensive reporting, the public may not be fully aware of the 
scope, progress, and results of agencies' planned analyses. Providing 
links or citations that point readers to documentation of the 
supporting analyses and data, as outlined in the executive orders and 
guidance, would help agencies to better adhere to executive 
directives, enhance the public transparency of retrospective analyses, 
and improve the usefulness of reporting. 

Independent Regulatory Agencies Followed a Different Retrospective 
Analysis Planning Process than Executive Agencies: 

Independent regulatory agencies were not required to develop 
retrospective review plans, but were encouraged to do so in Executive 
Order 13579 and a related memorandum providing additional guidance. 
[Footnote 38] Both of the independent regulatory agencies included 
within our scope, FCC and FTC, posted their final retrospective review 
plans on their agency websites. FCC also posted its final 
retrospective review plans in the Federal Register whereas FTC did 
not. Neither agency provided updates to the plans that identified the 
results or progress of their reviews, but are also not required to do 
so. However, both agencies said they have well-established practices 
to regularly review regulations and report outcomes. For example, FTC 
has a long-standing practice of maintaining a schedule, published 
annually, to ensure that each rule is reviewed approximately every 10 
years. The commission modifies the schedule to accelerate the review 
of rules when it observes economic or marketplace changes that 
necessitate a rule review. FCC provided additional explanation that it 
did not develop an updated plan partly due to transitions among the 
agency's top leadership. Specifically, the chairman position for the 
commission was vacant until November 4, 2013. 

We checked on planned analyses that FCC and FTC had identified in 
their plans and identified multiple completed analyses within the time 
period for our review. All of the two agencies' completed 
retrospective analyses that we reviewed resulted in changes to 
regulations, most often to improve the effectiveness of the 
regulations or make administrative changes. Of the 32 completed FCC 
analyses in our scope, 30 resulted in administrative changes, in 
particular cumulatively eliminating about 190 regulations that were 
obsolete. For example, FCC eliminated outdated and unnecessary 
reporting requirements related to international telecommunications 
traffic.[Footnote 39] Twenty of FCC's analyses focused on improving 
the effectiveness of the regulations, one focused on burden reduction, 
and two responded to statutory changes. Four of FTC's six completed 
analyses resulted in changes to improve the effectiveness of the 
regulations, four reduced burden, three addressed administrative 
changes, and two responded to statutory changes. Similar to executive 
agencies, these numbers are not mutually exclusive as some analyses 
led to multiple reported outcomes. For example, as part of its ongoing 
regulatory analysis of the Appliance Labeling Rule, FTC streamlined 
data reporting requirements for manufacturers, clarified testing 
requirements and enforcement provisions, improved online energy label 
disclosures, and made several minor technical changes and corrections. 
[Footnote 40] 

In a December 2013 report on Dodd-Frank Act regulations, we assessed 
financial regulatory agencies' plans to conduct retrospective reviews 
of existing rules.[Footnote 41] We found that federal financial 
regulators vary in their approaches and progress in developing and 
implementing plans to conduct retrospective reviews of their existing 
Dodd-Frank and other rules in recognition of Executive Order 13579. 
One of the independent financial regulatory agencies covered in that 
report, the Commodity Futures Trading Commission (CFTC) developed a 
final retrospective review plan. CFTC outlined a two-phase plan to 
conduct periodic retrospective reviews of its existing regulations in 
a June 2011 Federal Register notice. Under the first phase, CFTC 
reported that it has examined and revised a number of its existing 
regulations as part of its implementation of the Dodd-Frank Act. Under 
the second phase, CFTC plans to conduct retrospective reviews of the 
remainder of its regulations after substantial completion of its Dodd-
Frank Act rulemakings. CFTC provided OMB with periodic status reports 
on its retrospective review plan and reported in July 2013 that it is 
still in phase one. The Securities and Exchange Commission is in the 
process of developing a retrospective review plan. Other prudential 
regulators reported that they generally view their retrospective rule 
reviews conducted by statute or policy to be consistent with Executive 
Order 13579's principles and objectives. Thus, they did not develop a 
plan directly in response to the order. 

Agency Officials Identified Strategies and Barriers That Affected 
Their Implementation of Retrospective Analyses: 

Officials from the nine agencies that we selected to complete 
questionnaires and participate in our roundtable discussion cited many 
strategies and practices their agencies used as facilitators, or 
factors the agencies encountered as barriers, which affected their 
ability to implement key retrospective analysis requirements or 
guidance. The following sections highlight those facilitators and 
barriers that were most commonly reported by officials across the nine 
agencies. 

Agencies Identified Three Key Strategies to Facilitate Retrospective 
Analyses: 

Agency officials identified multiple strategies or practices that 
helped to facilitate their ability to conduct productive retrospective 
analyses. They most commonly identified three strategies or practices 
as being most significant in their experience. Two-thirds or more of 
the agencies cited each of the following three strategies as 
facilitators. 

Establish a Centrally Coordinated Review Process to Develop Review 
Plans: 

Six of the nine agencies reported that a standard centralized review 
process to develop retrospective review plans and track the progress 
of outcomes was a helpful strategy. Agencies often achieved this 
centralization through their policy analysis or general counsel's 
offices. In some instances, agencies also formed intradepartmental 
working groups.[Footnote 42] For example, DHS, which manages the 
retrospective analysis process through its general counsel's office, 
has an ongoing working group of representatives from CBP, Coast Guard, 
U.S. Citizenship and Immigration Services, Immigration and Customs 
Enforcement, the National Protection and Programs Directorate, 
Transportation Security Administration, and the Federal Emergency 
Management Agency. These representatives are responsible for 
conducting analyses within their component and building retrospective 
analysis into the agency culture. According to agency officials, the 
working group helps the department's components share best practices 
and learn from each other. FCC officials also said that centralizing 
their retrospective review effort in the general counsel's office, 
while seeking input from each bureau and office, is a key factor that 
is helping the agency to implement retrospective analyses. 

Leverage Existing Regulatory Activities to Identify Needed Changes: 

Six of the nine agencies reported that leveraging existing regulatory 
activities--such as regulatory planning, rulemaking, and enforcement 
processes--assisted them in implementing the executive orders and 
related guidance. Officials from most agencies reported that, when 
developing new rules, they examine existing regulations related to the 
rule as a normal course of conducting business. Agency officials 
expressed frustration at the misperception that they are not reviewing 
existing regulations, when in fact most of their regulatory activities 
involve such reviews. Agencies identified many examples of how they 
leveraged existing regulatory activities to develop their 
retrospective review plans in response to the executive orders. For 
example, the day-to-day regulatory enforcement activities of law 
enforcement staff informed which regulations to review and provided 
valuable information for assessing regulations. DOT reported that one 
of its agencies identified 15 to 20 regulations to review based on 
inspectors who noted consistent problems in the field, repeated 
questions from the public seeking clarification on certain 
regulations, and petitions for regulatory changes. Similarly, FTC 
reported that its law enforcement staff saw how regulations were 
working in the marketplace and provided ideas on whether to accelerate 
a review of a rule to address changes in the marketplace. These staff 
members were heavily involved in the regulatory review process and 
used their knowledge to inform how rules were working in practice. To 
help develop FCC's review plan, agency officials said that they used 
existing review processes, such as the statutorily required biennial 
review of regulations related to telecommunications regulations. 

Use Existing Feedback Mechanisms to Identify and Evaluate Regulatory 
Reforms: 

All of the agencies said that they used existing external feedback 
mechanisms to identify and evaluate regulatory reforms. In general, 
agency officials said that such mechanisms, which include petitions, 
listening sessions, and meetings with informed stakeholders, were more 
effective and useful than general public comments obtained through 
Federal Register notices. For example, one DHS official said that when 
commenting on the rulemaking process, members of the public tended to 
voice their opinions about general policy topics unrelated to the 
regulation being reviewed and, often, did not provide useful 
suggestions for regulatory changes. Some agency officials noted that 
the feedback they received from informed stakeholders, such as 
regulated entities, and policy advocacy groups, was more targeted and 
therefore more helpful for retrospective review purposes. 

Among other specific examples of useful feedback mechanisms, HHS 
officials with the Office of Policy and Planning within the Office of 
the National Coordinator for Health Information Technology said 
stakeholder feedback and emails requesting regulatory interpretations 
in a final rule were most useful because they helped them focus on 
areas of ambiguity or potential burden that could be reduced by 
altering or removing the regulatory requirement. One DOT official said 
the agency has advisory committees for almost all of its public safety 
regulatory areas. These committees include regulated entities and 
public safety advocates who regularly meet with DOT to recommend 
approaches to new, and revisions to, existing regulations at the front 
end of the process--usually before a proposed rule is published in the 
Federal Register. The DOT official said this helps the revision 
process run much more smoothly than it would otherwise. One FCC 
official said the agency has several active channels already in place 
to obtain public input in addition to the comments it receives from 
its commissioners, bureau chiefs, and advisory committees. Other 
agencies reported having similar feedback mechanisms that they 
considered to be most useful. 

Agencies Also Identified Other Helpful Strategies: 

A few of the agencies identified other helpful strategies for 
facilitating retrospective reviews. For example, the State Department 
requires its staff to submit a checklist to track the retrospective 
reviews of existing rules and reviews of proposed and final rules. 
This checklist served as an official record of the systematic review 
and addressed the elements of each significant rule. DHS said that 
compiling information about its regulations (for example, in a 
database), such as the normalized costs and benefits or the small 
entity impacts of each regulation, has been a useful tool. This 
information assisted the agency in identifying major cost drivers and 
regulations that were candidates for retrospective review and 
analysis. Another strategy cited by DOE was interagency discussion to 
identify and implement best practices adopted by other agencies. To 
obtain useful information from stakeholders, FTC adopted a 
standardized series of questions that provided a starting point to 
facilitate public comment on regulations subject to retrospective 
review. 

Persistent Barriers to Implementing Retrospective Analyses Remain: 

While agency officials representing the nine agencies participating in 
our roundtable identified multiple challenges or barriers to 
conducting retrospective reviews, the most commonly cited barriers 
identified as being most significant to their experience were (1) 
competing priorities and (2) difficulty obtaining sufficient data. The 
officials confirmed that these two factors have been persistent 
impediments to implementing retrospective analyses. This is consistent 
with barriers we found in our 2007 report.[Footnote 43] More than half 
of the agencies mentioned both of these factors as barriers to 
conducting retrospective reviews. 

Competing Priorities Hinder Agencies' Ability to Conduct Retrospective 
Analyses: 

Seven of the nine agencies cited competing priorities, particularly in 
a constrained budget environment, as a significant challenge to 
conducting retrospective regulatory analyses. Specifically, they 
referred to having a limited number of staff with the needed expertise 
to conduct retrospective analyses and other regulatory activities that 
are also among their agencies' priorities. For example, Commerce, DHS, 
and DOT officials reported that responding to new regulatory mandates 
imposed by Congress has been the focus for their staff, thereby 
limiting the amount of time and resources available to conduct 
retrospective analyses. DHS officials, in particular, pointed out that 
their agency cannot reassign existing staff to focus solely on 
retrospective reviews. HHS officials cited staff turnover across 
program areas and having limited staff with the necessary expertise to 
conduct retrospective analyses. According to agency officials, this 
issue can be exacerbated by requirements to report the same 
information multiple times to accomplish agencies' goals through 
duplicative processes. For example, in addition to conducting 
retrospective reviews, HHS's Centers for Medicare and Medicaid 
Services considers the need to eliminate or revise old, obsolete, or 
duplicative regulations when it reviews its plans quarterly for 
regulatory revisions or the need for new regulations. The agencies' 
feedback on this issue echoed the findings in our 2007 report. In that 
report, agencies cited lack of time and resources as the most critical 
barriers to their ability to conduct retrospective analyses because of 
the need to perform analyses simultaneously with other mission-
critical activities.[Footnote 44] This challenge of effectively 
balancing agency efforts among competing priorities reinforces the 
importance of the processes agencies use to select and prioritize 
which regulations to target for retrospective analysis. The challenge 
is magnified when considered in the context of a very extensive, and 
growing, inventory of existing regulations from which agencies must 
choose regulations to retrospectively assess within the limits of 
available resources. 

Agencies Reported Difficulty Obtaining Sufficient Data to Identify 
Improvements Attributed to Regulations and Other Barriers: 

Five of the nine agencies cited a lack of data to identify 
improvements attributed to regulations as a barrier to conducting 
reviews. In particular, agency officials cited challenges related to 
obtaining information from industry, the cost of data from outside 
sources, and the potential to transfer some data-collection burdens to 
the states, which face similar resource constraints. For example, DOT 
officials said that for their agency to have better data for some cost-
benefit analyses, it would have to put an additional burden on states 
to collect it, such as with police officers filling out longer 
accident forms to more accurately code how someone was injured. In 
addition, a DOT official said the data may be unavailable or too 
expensive to obtain, and it often can be against a regulated entity's 
interest to provide certain information. DHS said most of the data 
needed to conduct retrospective analyses of its regulations require 
original data collection efforts to obtain potentially business-
sensitive information from industry. According to DHS officials, 
restrictions under the Paperwork Reduction Act make it difficult to 
obtain information from a sufficient number of regulated entities to 
produce meaningful and useful results. A DOE official also said that 
additional data are constantly sought by its scientists and 
economists, and many data sets are expensive and have strict user 
requirements. Some agencies we spoke with noted, however, that at 
times they have more readily accessible existing data that can help 
inform their efforts to assess the performance of regulations. For 
example, to aid in its enforcement responsibilities, Commerce's Bureau 
of Industry and Security (BIS) uses U.S. export data already collected 
by the Census Bureau, which is another component within the 
department. One BIS official said these data help the agency conduct 
cost-benefit analyses and measure how their regulations are working. 
The barriers discussed above are consistent with findings from our 
2007 report, in which agencies reported they lacked the information 
and data needed to conduct reviews, such as baseline data for 
assessing regulations that they promulgated many years ago.[Footnote 
45] 

Agencies also confirmed other barriers that affected their ability to 
implement retrospective analyses, such as difficulty in (1) 
distinguishing between which results were produced by the regulation 
versus other factors, (2) resolving differences in opinion among 
internal or external stakeholders on the results, and (3) using the 
results to make regulatory decisions. For example, FTC reported that 
its ability to use analytical results of retrospective reviews to make 
changes to regulations were affected to a moderate extent by the 
difficulty in distinguishing between which results were produced by 
the regulation and which stemmed from other factors. In addition, FCC 
said its ability to use analytical results of retrospective reviews 
for the same purpose was affected to a moderate extent by the 
difficulty in resolving differences of opinion among internal or 
external stakeholders. Few agencies identified strategies to overcome 
the barriers discussed above. For example, the United States Patent 
and Trademark Office, within the Department of Commerce, combined two 
processes to leverage limited resources. The agency implemented 
retrospective review practices at the same time they were issuing new 
rules related to the Leahy Smith America Invents Act. In general, 
there was ultimately no consensus among the roundtable participants 
about strategies that might be more widely employed to effectively 
implement retrospective review while overcoming these barriers. 
However, later in this report we will discuss a related recommendation 
that we made to OMB in June 2013 to help address this matter. We 
recommended that the agency develop a detailed approach to examine 
such difficulties.[Footnote 46] 

Opportunities Exist to Improve Linkages between Regulatory Reviews and 
Priority Goals: 

Retrospective reviews are just one way, among others such as program 
evaluations, to help agencies evaluate their own performance. Given 
how important regulations are to the missions of regulatory agencies, 
the information generated from retrospective reviews can be leveraged 
to directly inform agency annual performance goals and measurement. 
Moreover, as discussed earlier, GPRAMA requires certain agencies, as 
determined by OMB, to develop APGs every 2 years. In addition to 
developing APGs, which are a subset of agencies' broader performance 
goals, agencies are required to identify the various regulations, as 
well as federal organizations and other program activities (both 
within and external to the agency) that contribute to each of their 
APGs, and review progress on a quarterly basis.[Footnote 47] This 
information is to be published on a governmentwide performance 
website, which OMB has implemented as Performance.gov, as well as in 
agencies' performance plans submitted to OMB.[Footnote 48] In April 
2013, we concluded that OMB's review process does not systematically 
check whether agencies have identified all relevant contributors to 
APGs. Thus, we recommended that the Director of OMB ensure that 
agencies adhere to OMB's guidance for website updates by providing 
complete information about regulations, among other activities, that 
contribute to each APG.[Footnote 49] Our review of the criteria 
agencies used for retrospective reviews shows they used multiple 
criteria, none of which explicitly mention ties to performance goals. 
[Footnote 50] Although the executive orders and guidance did not 
specifically direct agencies to include their broader performance 
goals, such as APGs, as a criterion, agencies may be missing a key 
opportunity to link these two activities. 

Agencies Reported Mixed Experiences Linking Retrospective Analyses to 
APGs: 

Agencies' responses about linkages between retrospective analyses and 
APGs were mixed. Three agencies (DOE, DHS, and DOT) said their 
recently completed, current, or future retrospective analyses were 
linked to their APGs to either a moderate or great extent. To 
illustrate this linkage, DOE said one of its APGs includes energy and 
costs savings as well as greenhouse gas reduction through the issuance 
of new or revised energy efficiency standards. As part of the agency's 
process for issuing revised standards, DOE said it considers 
retrospective analyses of the existing standards to determine if 
greater efficiency can be achieved at lower costs or with lower burden 
on the regulated community. Responses from the other agencies varied, 
generally stating that either (1) there was little to no connection 
between retrospective analyses and APGs or (2) agency officials did 
not know or had no basis to judge whether such linkages existed. 
[Footnote 51] In instances where multiple offices or bureaus within an 
agency completed the questionnaire, the responses also varied. For 
example, based on the responses we received from multiple offices 
within some agencies, it appears there is a mixed understanding or 
awareness of whether retrospective analyses are linked to APGs. This 
can be explained, in part, by the division of responsibilities within 
an agency where the office or staff responsible for carrying out 
retrospective analyses differs from the staff responsible for carrying 
out broader agency performance reviews. 

Although only three agencies in our review reported moderate or great 
linkages between their retrospective analyses and APGs, agency 
officials in offices from seven of the nine agencies identified 
regulations as key contributors to APGs. For example, while State 
reported little to no linkage between its retrospective analyses and 
APGs, the agency identified categories of regulations that contribute 
to three APGs, including "strengthening diplomacy and development by 
leading through civilian power." State's Bureau of Consular Affairs 
administers regulations that directly support this goal. DHS 
identified specific categories of regulations that contribute to its 
fiscal year 2013 APGs. For example, security regulations directly 
support the agency's APG to "strengthen aviation security 
counterterrorism capabilities by using intelligence-driven information 
and risk-based decisions." Immigration and border management 
regulations directly support DHS' APG to "improve the efficiency of 
the process to detain and remove criminal aliens." 

Most agencies cited linkages between retrospective analyses and agency 
goals other than APGs, such as agency strategic objectives or 
performance goals. However, the extent of these linkages varied. For 
example, Commerce's BIS said its retrospective analyses are tied to 
other agency goals to a great extent. During our roundtable 
discussion, one BIS official said the bureau's regulations directly 
support a department-level strategic objective. This linkage is 
documented in BIS' fiscal year 2015 budget submission and Commerce's 
fiscal year 2014-2018 strategic plan, which shows how BIS regulations 
contribute toward Commerce's strategic objective to increase U.S. 
exports by broadening and deepening the U.S. exporter base.[Footnote 
52] In addition, FTC said its ongoing retrospective review program 
ensures that FTC's rules target deceptive and unfair practices while 
also addressing changing market conditions and avoiding undue burdens 
on legitimate business activity. Although agencies identified examples 
where they established such linkages as described above, they may not 
be able to do so in all instances. Some agency officials we spoke with 
emphasized that regulations are only one contributor, among others, to 
broader agency performance goals being examined at higher levels 
within the agency. For example, one DOE official indicated there may 
not necessarily be a one-for-one linkage between an individual 
regulation and a single specific goal. Further, in a March 2004 
report, we found other difficulties regulatory agencies face in 
measuring performance including (1) obtaining data to demonstrate 
results, (2) accounting for factors outside of the agency's control 
that affect results, and (3) dealing with the long time periods often 
needed to see results.[Footnote 53] 

We asked agencies the extent to which they would like to receive more 
assistance or guidance on incorporating retrospective analyses into 
measuring and achieving agency goals, including APGs. Agencies 
expressed varying degrees of interest in receiving such assistance. In 
the cases where agencies received additional guidance on this matter, 
it was provided either internally within the individual agency or from 
OMB's OIRA. Only one of the nine agencies reported receiving 
additional assistance or guidance from other federal oversight offices 
and groups, such as the Performance Improvement Council (PIC). 
[Footnote 54] Given the common, long-standing difficulties agencies 
have continued to face in measuring the performance of various types 
of federal programs and activities, including regulations, we 
recommended in June 2013 that OMB work with the PIC to develop a 
detailed approach to examine these difficulties across agencies. This 
work would include identifying and sharing any promising practices 
from agencies that have overcome difficulties in measuring performance 
of these programs and activities.[Footnote 55] OMB staff agreed with 
this recommendation. As of March 2014, PIC staff told us they have 
taken initial steps to implement this recommendation through a pilot 
effort focused on acquisition, which is related to one of the areas 
highlighted in our June 2013 report. They said they plan to expand the 
model to focus on other issues, such as regulations and grants. 
According to PIC officials, it is too early in the process to have any 
supporting documentation related to these plans. We will continue to 
monitor progress on this effort. 

As noted earlier, one of the potential purposes for conducting 
retrospective analyses is to assess whether regulations, once 
implemented, achieved the expected benefits at the expected costs. 
However, the efforts of agencies included in our review to re-evaluate 
original cost-benefit analyses associated with their regulations 
varied. In response to our questionnaire, only three agencies reported 
that they often conduct reviews of regulations to determine whether 
the regulations are accomplishing the originally intended benefits at 
the expected cost. The other six agencies generally reported that they 
rarely, if ever, do this or did not know. Some agencies said they will 
sometimes revisit cost-benefit estimates to improve methods or models 
for conducting such analyses in the future.[Footnote 56] However, 
according to a few agency officials, they do not believe redoing past 
cost-benefit analyses is useful in the context of making decisions 
about individual regulations looking forward. In a broader context 
however, reexamining benefits and costs achieved after a regulation is 
implemented could provide data useful for performance reviews, 
including assessments of APGs, and is consistent with regulatory 
executive orders and guidance. One of the principles in Executive 
Order 13563 states that the regulatory system must measure, and seek 
to improve, the actual results of regulatory requirements. Subsequent 
guidance from the OIRA Administrator noted that this "points to the 
need for empirical assessment of the consequences of rules." 
Broadening the focus to think about retrospective reviews as a 
contributor to agency performance metrics related to APGs, rather than 
only in the context of examining a single regulation, may encourage 
agencies to revisit opportunities for more of this type of analysis. 

Opportunities Exist to Better Link Retrospective Analyses and Agency 
Priority Goals: 

The focus of retrospective analyses differs from agency performance 
reviews. Agencies' retrospective analyses tended to examine 
regulations on a micro level rather than their contribution to broader 
APGs. Regulations are developed and implemented individually, and 
often re-examined individually. In addition, the impetus for reviews 
is often statutory requirements, changes in the regulatory environment 
that affect the rule, availability of agency resources, or executive 
order requirements. The selection of which rule to review is typically 
tied to criteria such as the number of complaints or comments from 
regulated parties and the public, whether there have been changes in 
technology or other conditions since the regulation was issued, the 
potential for burden reduction, or the number of years since a 
regulation was promulgated or last reviewed, rather than whether it 
will be subject to upcoming performance goal discussions. The 
regulatory process and executive orders outlining how to conduct 
retrospective reviews have helped to shape the focus and timing of 
reviews. 

In contrast, agencies' performance reviews are designed to examine 
measures that cut across programs and policy tools at a macro level. 
For example, reviews of agency priority goals typically examine how 
agencies are accomplishing broad mission goals that may be implemented 
by a wide range of regulations, programs, grants, or other tools. As 
an illustrative example, in DOT's fiscal year 2014 performance plan, 
the agency organized the descriptions of its planned performance--
including its agency priority goals--into broad themes under its 
strategic goals. As shown in figure 3, DOT identified the regulations 
and enabling legislation that contribute to its APG for aviation 
safety. 

Figure 3: DOT Example of How Regulations Link to an Agency Performance 
Goal: 

[Refer to PDF for image: illustration] 

Document sections indicating internal contributors to the agency 
priority goal: 

Aviation Safety (FAA): 

Why Is This Effort Necessary? 

The Federal Aviation Administration (FAA) will enhance safety by 
continuing to reinforce a positive safety culture and by making 
improvements to our aviation standards and industry oversight. 

Reducing the risk of aviation accidents is a DOT Agency Priority Goal 
that concentrates on making improvements to General Aviation (GA) and 
Commercial Aviation fatal accident rates. 

Strategic Outcome and Supporting Performance Measures: 

Reduction in transportation-related fatalities. 

Agency Priority Goal: Reduce risk of aviation accidents. By September 
30, 2013, reduce aviation fatalities by addressing risk factors both 
on the ground and in the air. Commercial Aviation (i.e., airlines): 
Reduce the rate of fatalities to no more than 7.4 per 100 million 
people on board in fiscal year (FY) 2013. 

General Aviation (i.e., private planes): Reduce fatal accident rate 
per 100,000 flight hours to no more than 1.06 in FY 2013. 

Reduce Commercial Aviation Air Carrier Fatalities to No More Than 7.2 

Reduce the General Aviation Fatal Accident Rate Per 100,000 Flight 
Hours to No More Than 1.05 in FY 2014. 

Enabling Legislation and Regulations: 

Commercial Aviation and General Aviation Safety: 

* Airline Safety and Federal Aviation Administration Extension Act of 
2010, P.L. 111-216;
* Airport and Airway Extension Act of 2011, Part II, P.L.112-16;
* FAA Modernization and Reform Act of 2012, P.L. 112-95;
* Title 14 CFR Part 39 Airworthiness Directives; and,
* Title 49, U.S.C., Transportation. 

Runway Safety: 
* FAA Advisory Circular 150/5300-Airport Series; and,
* Title 14 CFR Aeronautics and Space:
- Part 61—-Certification: Pilots, Flight Instructors, and Ground 
Instructors; 
- Part 91—-General Operating and Flight Rules; 
- Part 121—-Operating Requirements: Domestic, Flag, and Supplemental 
Operations; 
- Part 135-—Operating Requirements: Commuter and On Demand Operations 
and Rules Governing Persons on Board Such Aircraft; 
- Part 139-—Certification of Airports n Part 141—Pilot Schools; 
- Part 142-—Training Centers. 

Source: GAO analysis of the Department of Transportation Fiscal Year 
2014 Annual Performance Plan. 

[End of figure] 

Given the specific focus of retrospective regulatory analyses, several 
agency officials said the staff conducting the analyses often oversee 
implementation of the regulation and are not involved in, or sometimes 
aware of, performance discussions that are held at higher levels of 
the agency. Similar to the earlier discussion about barriers, agency 
officials said it may sometimes be difficult to isolate the effect of 
the regulation versus other external factors contributing to the same 
outcome. Including consideration of APGs as another part of planning 
and reporting retrospective reviews could strengthen these linkages. 
Currently, the selection of regulations to review is generally driven 
by a bottom-up process, while performance reviews are driven by a top-
down process. While both processes have a useful purpose and 
structure, retrospective reviews could help inform broader performance 
discussions if agencies were to better link (1) the selected 
regulations to review, (2) the timing of reviews, and (3) the 
information assessed. First, as previously noted, the selection of 
rules to review is typically tied to specific criteria. Including 
whether a regulation contributes to an APG expected to be reviewed by 
management as one of the criteria for prioritizing reviews could help 
retrospective analyses contribute useful information to the discussion 
of whether and how goals are being accomplished. Second, better 
alignment between the timing of when regulations are reviewed and when 
APGs related to those regulations are examined by agency leadership 
would help to ensure that evaluations are conducted when the 
information would be most useful. Third, examining regulations that 
collectively contribute to a specific APG could also improve the 
usefulness of retrospective reviews. As illustrated in figure 3, 
multiple DOT regulations contributed to the agency's APG on aviation 
safety. 

Greater attention to the cumulative effects of regulations would also 
help agencies to better address certain key analytical and management 
principles. First, this would be consistent with Executive Order 
13610's direction that agencies consider the cumulative effects of 
their regulations, among other priorities. In addition, as agencies 
implement their APGs under GPRAMA, they are required to identify and 
assess the various contributors to those goals, which can include 
multiple regulations. Information generated from retrospective 
analyses can serve as one source of valuable information about such 
cumulative effects. Finally, this would help agencies to meet the 
internal control standard for information and communications. Among 
other things, the standard states that program managers need 
operational data to determine whether the agency is meeting its 
strategic and annual performance goals for accountability and for the 
effective and efficient use of resources.[Footnote 57] Cumulative 
analysis could include joint reviews with other agencies as 
appropriate. Four agencies we spoke with reported conducting joint 
reviews with other agencies. For example, HHS officials said the 
Centers for Disease Control and Prevention, a division within HHS, co-
regulates a shared list of select agents and toxins with the Animal 
Plant Health and Inspection Service (APHIS) within the U.S. Department 
of Agriculture. According to HHS, APHIS provides input during related 
retrospective reviews of these regulations. This type of joint review 
can provide more comprehensive and useful information about the 
collective impact of the rules on performance and on the regulated 
community. 

To better position agencies to conduct future retrospective analyses 
that could contribute to performance reviews, it is important for 
agencies to have considered in advance how they will evaluate their 
regulations. In 2007, we recommended that during the promulgation of 
certain new rules (such as significant rules) agencies consider 
whether and how they will measure the performance of the regulation, 
including how and when they will collect, analyze, and report the data 
needed to conduct a retrospective review. Consistent with that 
recommendation, OMB guidance on implementing retrospective analysis 
states that future regulations should be designed and written in ways 
that facilitate evaluation of their consequences and thus promote 
retrospective analyses. OMB asked agencies to identify in their 
retrospective review plans what metrics the agencies will use to 
evaluate regulations after they have been implemented. We asked OIRA 
the extent to which agencies are implementing that guidance. One OIRA 
official said his office has seen some evidence of agencies writing 
rules in ways that promote future retrospective analyses. He cited a 
December 2013 final rule by Commerce's National Oceanic and 
Atmospheric Administration regarding regulations to reduce the threat 
of ship collisions with North Atlantic right whales.[Footnote 58] The 
final rule removed an expiration date (or "sunset clause") contained 
in the regulations but included provisions for the agency to review 
the costs and benefits of the rule on a periodic basis, as required by 
Executive Order 13563. The agency expects to conduct such a review no 
later than 5 years from the publication of the final rule. The rule 
also discussed data and metrics the agency could use as measures of 
effectiveness. Ensuring that agencies build in such performance 
metrics and a timeline for evaluating regulations after implementation 
would not only help facilitate retrospective analyses, but also help 
to lay a foundation to more closely tie retrospective analyses to 
reviews of broader agency priority goals. Moreover, GPRAMA's 
requirements for agencies to identify and assess how their various 
programs and activities, including regulations, contribute to agency 
performance goals and APGs further underscore the need for agencies to 
take such action. 

Conclusions: 

We have long advocated the potential usefulness to Congress, agencies, 
and the public of conducting retrospective regulatory analyses while 
recognizing the potential difficulties of doing so. From January 2011 
through August 2013, executive agencies identified more than 650 
initiatives (planned retrospective regulatory analyses). Of these 
initiatives, 246 resulted in agencies implementing some type of final 
action by August 31, 2013. The majority of the completed reviews have 
led to regulatory changes focused on outcomes such as reducing burden 
and improving the effectiveness of regulations. While OMB has provided 
a sample template for agencies to use for reporting on the progress 
and the results of retrospective analyses, it could do more to enhance 
the transparency and usefulness of the information provided to the 
public. Although agencies posted their retrospective review plans 
online, obtaining a comprehensive picture of agencies' progress was 
difficult when results were spread across multiple plans. In addition, 
consistently providing links or citations to the supporting analyses 
and data, and including more detail on the methodologies and key 
assumptions used to estimate savings, would help Congress and the 
public to better understand the basis for projected results. 

Given how important regulations are to the missions of regulatory 
agencies, the information generated from retrospective analyses can 
also be leveraged to inform APGs, which are a subset of agencies' 
broader performance goals. The broader perspective and focus of 
performance management and reviews may also enhance evaluation of the 
effectiveness of regulations beyond what is possible through 
retrospective analysis alone. This is especially true given the 
practical limits to the number of analyses that could be done compared 
to the large and growing inventory of regulations. We recognize that 
agencies conduct retrospective analyses for many reasons. However, 
contributing to evaluation of agency priority goals could also be 
among those reasons. Therefore, agencies should at least consider 
whether opportunities exist to inform performance reviews when 
selecting which regulations to retrospectively review. Among the 
agencies that participated in our roundtable, few identified examples 
where their retrospective review analyses were linked to APGs. Steps 
that agencies can take to strengthen the linkages between 
retrospective reviews and performance reviews include identifying 
specific regulations that contribute to APGs and including whether a 
regulation contributes to a key performance goal expected to be 
reviewed by management as one of the criteria for prioritizing 
retrospective reviews. Doing so may also provide greater impetus for 
agencies to review the benefits and costs of existing regulations as 
part of their retrospective analyses. Many of the agencies we spoke 
with reported rarely doing so in the context of reviewing individual 
regulations, and some agencies saw little value to the investment. 
However, there would be more incentive to measure benefits and costs 
if retrospective analyses were viewed in the broader context of 
providing information on the actual, rather than projected, 
performance and results of regulatory programs and agencies. This 
would also reinforce existing guidance that agencies should identify 
how they will measure the performance of new significant regulations 
when originally published. 

The executive branch has already incorporated many, but not all, of 
our prior relevant recommendations into current guidance. Some of the 
opportunities to improve reporting on outcomes of retrospective 
analyses and strengthen linkages between these analyses and APGs could 
be implemented through augmenting existing guidance. However, 
additional opportunities for improvement depend in part on efforts to 
ensure that agencies are consistently held accountable for 
implementing existing guidance. 

Recommendations for Executive Action: 

To improve agencies' retrospective regulatory review processes and 
reporting, and strengthen linkages between retrospective reviews and 
agency performance management, we recommend the Director of the Office 
of Management and Budget direct the Administrator of the Office of 
Information and Regulatory Affairs to take the following three actions: 

1. Work with regulatory agencies to implement existing guidance, and 
update guidance where needed, to improve the reporting of outcomes in 
their retrospective regulatory review plans by taking actions such as: 

* publishing a link to updated plans, which list recent results and 
anticipated outcomes, on the White House website; 

* submitting evidence that agencies listed updates of their plans on 
their "Open Government" web pages; 

* providing more comprehensive information on completed reviews in 
agencies' most recent plans and progress reports by (1) ensuring the 
most recent published plan contains a complete accounting of all 
completed reviews rather than expecting readers to review multiple 
plans, and (2) including the supporting analysis and data for results 
by listing a link or citation to the related documentation. 

2. Ensure that the contributions made by regulations toward the 
achievement of APGs are properly considered and improve how 
retrospective regulatory reviews can be used to help inform 
assessments of progress toward these APGs by directing in guidance 
that agencies take such actions as: 

* identifying whether a regulation contributes to an APG expected to 
be reviewed by management as one of the criteria for prioritizing 
retrospective analyses and for the timing of these analyses; and: 

* once an agency prioritizes a retrospective analysis based, in part, 
on its support of an APG, improving the usefulness of that analysis by 
examining regulations that collectively contribute to the goal in the 
scope of the review as appropriate. 

3. Ensure that OIRA, as part of its oversight role, monitor the extent 
to which agencies have implemented the guidance on retrospective 
regulatory review requirements outlined in the related executive 
orders and confirm that agencies have identified how they will assess 
the performance of regulations in the future. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to the Director of the Office of 
Management and Budget, the Secretaries of Commerce, Energy, Health and 
Human Services, Homeland Security, State, and Transportation, as well 
as the Administrator of the Environmental Protection Agency and the 
Commissioners of the Federal Communications Commission and Federal 
Trade Commission. 

In oral comments received on April 3, 2014, staff from OMB's Office of 
Information and Regulatory Affairs (OIRA) generally agreed with the 
recommendations in this report. In response to our first 
recommendation, OIRA staff generally agreed but said they were open to 
taking actions other than updating the White House website in light of 
a decision to discontinue posting agencies' updates as individual 
agencies assume primary responsibility for posting updates on their 
"Open Government" web pages. OIRA staff were supportive of identifying 
ways to improve agencies' posting and reporting of retrospective 
analyses and progress updates. In response to our second 
recommendation, OIRA staff agreed and said the agency is currently 
working on strategies to help facilitate agencies' ability to consider 
and improve how retrospective reviews can be used to help inform APGs, 
and otherwise be better integrated into agency strategic decision 
making. In response to our third recommendation, OIRA staff agreed, 
emphasizing that this remains a priority and therefore they would 
continue to monitor the extent to which agencies implement the 
guidance on retrospective regulatory review requirements. Further, 
OIRA staff said that as part of its review of agencies' rules, unified 
agendas, and regulatory plans, OIRA will continue to encourage 
agencies to identify beforehand how they will evaluate the 
effectiveness of a regulation after it has been put in place. OIRA 
staff also provided technical comments and clarifications, which we 
incorporated as appropriate. 

The Departments of Commerce, Homeland Security, Health and Human 
Services, and State, the Environmental Protection Agency, and the 
Federal Trade Commission provided technical comments, which we 
incorporated as appropriate. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the Director of the Office of Management and Budget, the 
Secretaries of Commerce, Energy, Health and Human Services, Homeland 
Security, State, and Transportation, as well as the Administrator of 
the Environmental Protection Agency and the Commissioners of the 
Federal Communications Commission and Federal Trade Commission. We are 
also sending copies of this report to relevant congressional 
committees. In addition, the report is available at no charge on GAO's 
website at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-6806 or sagerm@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who have made contributions to 
this report are listed in appendix II. 

Signed by: 

Michelle Sager: 
Director, Strategic Issues: 

[End of section] 

Appendix I: Key Retrospective Analysis Requirements Listed in 
Executive Orders and Memorandums Issued Since January 2011: 

Table 2 provides a summary of the various executive orders and related 
memorandums that were issued since January 2011 related to agencies' 
retrospective regulatory review plans and analyses. For presentation 
purposes, we highlighted the elements of these directives that relate 
to (1) planning retrospective analyses, (2) implementing these plans 
and analyses, and (3) reporting the results of completed analyses. 

Table 2: Executive Orders and Memorandums Issued Since January 2011 
Related to Retrospective Regulatory Analysis: 

Planning Retrospective Regulatory Analyses: 

Develop retrospective review plans that describe progress, anticipated 
accomplishments, and proposed timelines for relevant regulatory reform 
actions; 
Executive Orders: 
* Exec. Order 13563, "Improving Regulation and Regulatory Review"; 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens"; 
* Exec. Order 13579 (for independent agencies),; 
* "Regulation and Independent Regulatory Agencies"; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-19, "Retrospective Analysis of Existing Significant 
Regulations," (Apr. 25, 2011); 
* M-11-25, "Final Plans for Retrospective Analysis of Existing Rules," 
(June 14, 2011); 
* Implementation of Retrospective Review Plans," (Oct. 26, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Seek the views of the public on retrospective review plans; 
Executive Orders: 
* Exec. Order 13563, "Improving Regulation and Regulatory Review"; 
* Exec. Order 13610,; 
* "Identifying and Reducing Regulatory Burdens"; 
* Exec. Order 13579 (for independent agencies), "Regulation and 
Independent Regulatory Agencies"; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-25, "Final Plans for Retrospective Analysis of Existing Rules," 
(June 14, 2011); 
* "Implementation of Retrospective Review Plans," (Oct. 26, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Submit updated retrospective review plans semiannually to the Office 
of Information and Regulatory Affairs and release them to the public; 
Executive Order: 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens"; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-19, "Retrospective Analysis of Existing Significant 
Regulations," (Apr. 25, 2011); 
* M-11-25, "Final Plans for Retrospective Analysis of Existing Rules," 
(June 14, 2011); 
* Implementation of Retrospective Review Plans," (Oct. 26, 2011). 

Conducting Retrospective Regulatory Analyses: 

Conduct a quantifiable assessment of the current costs and benefits of 
any proposed changes to existing regulations to the extent possible; 
Executive Orders: 
* Exec. Order 13563, "Improving Regulation and Regulatory Review"; 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens" 
(give priority to reviews with quantifiable costs and benefits); 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-25, "Final Plans for Retrospective Analysis of Existing Rules," 
(June 14, 2011); 
* Implementation of Retrospective Review Plans," (Oct. 26, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Give consideration to the cumulative effects of their own regulations, 
including cumulative burdens, and to the extent practicable and 
consistent with law give priority to reforms that would make 
significant progress in reducing those burdens while protecting public 
health, welfare, safety, and our environment; 
Executive Orders: 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens"; 
* Exec. Order 13563, "Improving Regulation and Regulatory Review" 
(indirectly related--consideration of cumulative effects of 
regulations when doing analysis in general); 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Seek the views of the public on retrospective review analysis; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Vest responsibility for retrospective review with a high-level agency 
official who can secure cooperation across the agency; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Maintain sufficient independence from the offices responsible for 
writing and implementing regulations; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Coordinate the development of the new retrospective plans with other 
retrospective review requirements, such as the Regulatory Flexibility 
Act, 5 U.S.C. §610; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Reporting Results of Retrospective Analyses: 

Provide specific timelines for implementation of reforms based on 
retrospective reviews, to the extent possible, particularly focusing 
on high-priority reforms that promise significant cost savings; 
Executive Order: 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens"; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-19, "Retrospective Analysis of Existing Significant 
Regulations," (Apr. 25, 2011). 

Release retrospective analyses and their supporting data to the public 
by posting them online whenever possible; 
Executive Orders: 
* Exec. Order 13563, "Improving Regulation and Regulatory Review"; 
* Exec. Order 13610, "Identifying and Reducing Regulatory Burdens"; 
* Exec. Order 13579 (for independent agencies), "Regulation and 
Independent Regulatory Agencies"; 
Executive Memorandums: 
* M-11-10, Executive Order 13563, "Improving Regulation and Regulatory 
Review," (Feb. 2, 2011); 
* M-11-19, "Retrospective Analysis of Existing Significant 
Regulations," (Apr. 25, 2011); 
* "Implementation of Retrospective Review Plans," (Oct. 26, 2011); 
* M-11-28, Executive Order 13579, "Regulation and Independent 
Regulatory Agencies," (July 22, 2011). 

Source: GAO analysis of executive orders and executive memorandums. 

[End of table] 

[End of section] 

Appendix II: Staff Contact and Acknowledgments: 

GAO Contact: 

Michelle Sager, (202) 512-6806 or sagerm@gao.gov. 

Staff Acknowledgments: 

In addition to the contact named above, Tim Bober, Assistant Director, 
Leah Q. Nash, Latesha Love, Lou V.B. Smith, and Wesley Sholtes made 
major contributions to this report. Tom Beall, Tim Guinane, Andrea 
Levine, Benjamin T. Licht, Donna Miller, Cindy Saunders, Stephanie 
Shipman, and Stewart Small also made key contributions to this report. 

[End of section] 

Footnotes: 

[1] In addition to regulations, other tools of government include 
contracts, grants, direct services, tax expenditures, and research and 
development. 

[2] Significant regulatory actions are those that may have an annual 
effect on the economy of $100 million or more, among other things. 

[3] GAO, Reexamining Regulations: Opportunities Exist to Improve 
Effectiveness and Transparency of Retrospective Reviews, [hyperlink, 
http://www.gao.gov/products/GAO-07-791] (Washington, D.C.: July 16, 
2007). 

[4] There is no one standard term or definition for the variety of 
activities that might be considered retrospective regulatory analysis. 
For example, in various contexts, these activities have been referred 
to as retrospective reviews or look-backs. In this report, we use 
terminology consistent with Executive Orders 13563, 13579, and 13610. 
We refer to the general plans and updates that agencies prepared in 
response to the executive orders as "retrospective review plans" and 
to the agencies' individual analyses of specific existing regulations 
as "retrospective analyses." 

[5] "Executive agencies" are cabinet departments and other agencies 
that answer directly to the President. "Independent regulatory 
agencies" refer to the boards and commissions identified as such in 
the Paperwork Reduction Act, for example, the Securities and Exchange 
Commission (SEC). 44 U.S.C. § 3502(5). 

[6] Government Performance and Results Act of 1993, Pub. L. No. 103-
62, 107 Stat. 285 (Aug. 3, 1993). GPRA Modernization Act of 2010, Pub. 
L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011). 

[7] We selected executive agencies based on the number of published 
final rules. To avoid duplication of effort, we did not include as 
part of our review the federal financial regulators, such as the 
Commodity Futures Trading Commission, whose retrospective review plans 
and activities were addressed in our December 2013 report on the 
implementation of the Dodd-Frank Act. We have, however, summarized 
relevant findings from that report where appropriate. See GAO, Dodd-
Frank Regulations: Agencies Conducted Regulatory Analyses and 
Coordinated but Could Benefit from Additional Guidance on Major Rules, 
[hyperlink, http://www.gao.gov/products/GAO-14-67] (Washington, D.C.: 
Dec. 11, 2013). 

[8] We did not include cases where the agency had completed a 
retrospective analysis but had not implemented a final action by 
August 31, 2013--for example when an agency had only issued a proposed 
rule by our cut-off date. 

[9] The Unified Agenda, which is to be published every spring and 
fall, provides uniform reporting of data on regulatory and 
deregulatory activities under development throughout the federal 
government, covering approximately 60 departments, agencies, and 
commissions. Each edition of the Unified Agenda includes regulatory 
agendas from all federal entities that currently have regulations 
under development or review. The Regulatory Plan, which is published 
as part of the fall edition of the Agenda, identifies regulatory 
priorities and contains additional detail about the most important 
significant regulatory actions that agencies expect to take in the 
coming year. 

[10] For three of the agencies, multiple offices or bureaus within the 
agency completed the questionnaire. 

[11] The CFR annual edition is the codification of the general and 
permanent rules published in the Federal Register by the departments 
and agencies of the federal government. 

[12] For example, Section 610 of the Regulatory Flexibility Act and 
Section 5 of Executive Order 12866 both require agencies to 
periodically review certain existing regulations. 

[13] For example, on EPA's web page, in which the agency provides 
information to the public on the status of priority rulemakings and 
reviews of existing regulations, it states that of the agency's 
current regulatory workload, almost two-thirds is a review of an 
existing regulation. 

[14] 76 Fed. Reg. 3821 (Jan. 21, 2011). 

[15] 76 Fed. Reg. 41,587 (July 14, 2011). Independent regulatory 
agencies are not required to comply with executive orders. 

[16] 77 Fed. Reg. 28,469 (May 14, 2012). 

[17] 31 U.S.C. § 1120(b). The 24 CFO Act agencies, generally the 
largest federal agencies, are the Departments of Agriculture, 
Commerce, Defense, Education, Energy, Health and Human Services, 
Homeland Security, Housing and Urban Development, the Interior, 
Justice, Labor, State, Transportation, the Treasury, and Veterans 
Affairs, as well as the U.S. Agency for International Development, 
Environmental Protection Agency, General Services Administration, 
National Aeronautics and Space Administration, National Science 
Foundation, Nuclear Regulatory Commission, Office of Personnel 
Management, Small Business Administration, and Social Security 
Administration. (31 U.S.C. § 901(b)). 

[18] 31 U.S.C. §§ 1115(b)(5)(C), 1122(b)(3)(C). OMB is required to 
develop a single, government-wide performance website to communicate 
government-wide and agency performance information. The website--
implemented by OMB as Performance.gov--is required to make available 
information on APGs and cross-agency priority goals, updated on a 
quarterly basis; agency strategic plans, annual performance plans, and 
annual performance reports; and an inventory of all federal programs. 

[19] GAO, Managing for Results: Agencies Should More Fully Develop 
Priority Goals under the GPRA Modernization Act, [hyperlink, 
http://www.gao.gov/products/GAO-13-174] (Washington, D.C.: Apr. 19, 
2013). 

[20] As of April 3, 2014, OMB had not provided a status update for 
this open recommendation. 

[21] There were also instances where agencies completed reviews, but 
had not implemented a final action by August 31, 2013. For example, 
the Department of Justice (DOJ) reviewed regulations administered by 
the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) to 
extend the term of import permits for firearms, ammunition, and 
defense articles from 1 year to 2 years. According to DOJ, the 
additional time will allow importers sufficient time to complete the 
importation of the authorized commodity and is expected to result in a 
substantial cost and time savings for both the industry and ATF. The 
agency had projected issuing the rule in November 2013 but did not do 
so. 

[22] 78 Fed. Reg. 6027 (Jan. 29, 2013). 

[23] The other two instances where agencies determined no additional 
action was needed appeared in updates of DOT and HHS retrospective 
review plans. 

[24] 77 Fed. Reg. 21,312 (Apr. 9, 2012). 

[25] [hyperlink, http://www.gao.gov/products/GAO-07-791]. 

[26] 77 Fed. Reg. 64,237 (Oct. 19, 2012). 

[27] 76 Fed. Reg. 23,177 (Apr. 26, 2011). 

[28] 78 Fed. Reg. 40,945 (July 9, 2013). 

[29] A June 2012 memorandum from the Administrator of OIRA identified 
nine types of steps (such as eliminating redundant or unnecessary 
collections, simplified applications, and use of electronic 
communications) that agencies could take to reduce paperwork and 
reporting burdens, consistent with Executive Order 13610 and the 
Paperwork Reduction Act. 

[30] In practice, agencies reported the progress of their 
retrospective analyses in updated "retrospective review plans." For 
discussion purposes in this report, we may use the term "progress 
reports" or "updates" to clarify when we are referring to agencies' 
plans that contain progress reports or updates on their retrospective 
analyses versus the agencies' original review plans. 

[31] Agencies also anticipated quantified cost savings for some of the 
completed reviews that we did not include in our analysis because 
agencies had not yet taken final actions. For example, in August 2013 
DOT proposed a rule regarding driver vehicle inspection reports filed 
by truck drivers that the agency estimates could save $1.7 billion 
annually in time and paperwork burdens. 78 Fed. Reg. 48,125 (Aug. 7, 
2013). DOT expects to issue the final rule in November 2014. 

[32] 77 Fed. Reg. 29,002 and 29,034 (May 16, 2012). 

[33] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[34] 77 Fed. Reg. 53,967 (Sept. 4, 2012). 

[35] As part of the President's Open Government Directive in 2009, OMB 
directed agencies to create an "Open Government" web page to serve a 
variety of purposes and as a step toward the goal of creating more 
open government. 

[36] As of March 31, 2014, that site [hyperlink, 
http://www.whitehouse.gov/21stcenturygov/actions/21st-century-
regulatory-system] listed only the initial final plans completed and 
updates through May 2012. 

[37] In April 2014, OIRA staff told us they believe that all of the 
executive agencies have now posted their updates on their own "Open 
Government" web pages, but acknowledged that this information may not 
always be easy to find. 

[38] OMB provided additional guidance to independent regulatory 
agencies in memorandum M-11-28, Executive Order 13579, "Regulation and 
Independent Regulatory Agencies," issued on July 22, 2011. 

[39] 76 Fed. Reg. 42,567 (July 19, 2011). 

[40] 78 Fed. Reg. 2200 (Jan. 10, 2013). 

[41] [hyperlink, http://www.gao.gov/products/GAO-14-67]. 

[42] For additional information on interagency groups and 
collaboration, see GAO, Managing for Results: Implementation 
Approaches Used to Enhance Collaboration in Interagency Groups, 
[hyperlink, http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: 
Feb. 14, 2014). 

[43] [hyperlink, http://www.gao.gov/products/GAO-07-791]. 

[44] To help address this issue, we recommended that agency policies 
and guidance include defined selection criteria for prioritization of 
retrospective analyses, which was later reflected in the executive 
orders and OMB guidance. 

[45] As discussed in the background section of the report, we 
previously recommended--and OMB subsequently incorporated in guidance 
to agencies--that agencies should consider how they will measure the 
performance of new regulations when issued. 

[46] GAO, Managing for Results: Executive Branch Should More Fully 
Implement the GPRA Modernization Act to Address Pressing Governance 
Challenges, [hyperlink, http://www.gao.gov/products/GAO-13-518] 
(Washington, D.C.: June 26, 2013). 

[47] The two independent regulatory agencies that participated in our 
roundtable discussions--FCC and FTC--were not selected by OMB to 
develop APGs. However, they are to develop annual performance goals 
and identify the various contributors to each of those goals. 

[48] 31 U.S.C. §§ 1115(b)(5)(C), 1122(b)(3)(C). 

[49] [hyperlink, http://www.gao.gov/products/GAO-13-174]. As of April 
3, 2014, OMB had not provided a status update for this open 
recommendation. 

[50] However, two agencies reported that when setting priorities for 
their retrospective analyses, they consider whether regulations are 
achieving their intended outcomes. Another agency cited a connection 
between its retrospective analyses and strategic planning efforts. 

[51] Responses from the two independent regulatory agencies we spoke 
with--FCC and FTC--reflected that they are not required to develop 
APGs under GPRAMA. 

[52] According to BIS' fiscal year 2015 budget submission, the agency 
is not a leader or a participant in any APGs. 

[53] See for example GAO, Results-Oriented Government: GPRA Has 
Established a Solid Foundation for Achieving Greater Results, 
[hyperlink, http://www.gao.gov/products/GAO-04-38] (Washington, D.C.: 
March 10, 2004). 

[54] We also asked agencies to identify any assistance they received 
from two additional federal oversight entities--the Regulatory Working 
Group and OMB's Office of Performance and Personnel Management. PIC is 
responsible for assisting OMB in improving federal government 
performance and achieving cross-agency priority goals. Among its 
responsibilities, the council is to facilitate the exchange among 
agencies of useful performance improvement practices and work to 
resolve government-wide or crosscutting performance issues. 31 U.S.C. 
§§ 1124(b)(2)(D),(E). 

[55] [hyperlink, http://www.gao.gov/products/GAO-13-518]. 

[56] EPA's retrospective review plan includes a retrospective study of 
the costs of EPA regulations to evaluate whether cost estimates 
developed before and after regulations were implemented differ 
substantially. The results may lead to the agency improving its cost 
estimation methodology. This study was not completed within the time 
frame covered by our review. Further, EPA officials said their 
retrospective reviews usually do examine whether rules are achieving 
the intended benefits and that compliance cost is considered when 
developing revised rules. 

[57] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[58] 78 Fed. Reg. 73,726 (Dec. 9, 2013). 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
"E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]