Learning from Experience: An Assessment of the Retrospective Reviews of Agency Rules
and the Evidence for Improving the Design and Implementation of Regulatory Policy
Joseph E. Aldy
2014
RPP-2014-28
Regulatory Policy Program
Mossavar-Rahmani Center for Business and Government Harvard Kennedy School
79 John F. Kennedy Street, Weil Hall Cambridge, MA 02138
http://www.ksg.harvard.edu/cbg/rpp/home.htmhttp://www.ksg.harvard.edu/cbghttp://www.ksg.harvard.edu/
CITATION
This paper may be cited as: Aldy, Joseph E. 2014. “Learning from Experience: An Assessment of the Retrospective Reviews of Agency Rules and the Evidence for Improving the Design and Implementation of Regulatory Policy.” Regulatory Policy Program Working Paper RPP-2014-28. Cambridge, MA: Mossavar-Rahmani Center for Business and Government, Harvard Kennedy School, Harvard University. Comments may be directed to the author.
REGULATORY POLICY PROGRAM The Regulatory Policy Program at the Mossavar-Rahmani Center for Business and Government serves as a catalyst and clearinghouse for the study of regulation across Harvard University. The program's objectives are to cross-pollinate research, spark new lines of inquiry, and increase the connection between theory and practice. Through seminars and symposia, working papers, and new media, RPP explores themes that cut across regulation in its various domains: market failures and the public policy case for government regulation; the efficacy and efficiency of various regulatory instruments; and the most effective ways to foster transparent and participatory regulatory processes. The views expressed in this paper are those of the author and do not imply endorsement by the Regulatory Policy Program, the Mossavar-Rahmani Center for Business and Government, Harvard Kennedy School, or Harvard University.
FOR FURTHER INFORMATION Further information on the Regulatory Policy Program can be obtained from the Program's executive director, Jennifer Nash, Mossavar-Rahmani Center for Business and Government, Weil Hall, Harvard Kennedy School, 79 JKF Street, Cambridge, MA 02138, telephone (617) 495-9379, telefax (617) 496-0063, email [email protected]. The homepage for the Regulatory Policy Program can be found at: http://www.hks.harvard.edu/centers/mrcbg/programs/rpp
mailto:[email protected]
1
Learning from Experience: An Assessment of the Retrospective Reviews of Agency Rules and the Evidence for Improving the Design and Implementation of Regulatory Policy
Joseph E. Aldy
Harvard Kennedy School
Resources for the Future
National Bureau of Economic Research
November 17, 2014
2
Contact Information, Disclosure, and Acknowledgments Joseph E. Aldy is an Assistant Professor of Public Policy, John F. Kennedy School of Government, Harvard University; a Visiting Fellow, Resources for the Future; and a Faculty Research Fellow, National Bureau of Economic Research. His contact information is: e: [email protected]; i: http://www.hks.harvard.edu/fs/jaldy/; v: 617-496-7213; m: Mailbox 57, John F. Kennedy School of Government, 79 JFK Street, Cambridge, MA 02138. This report was prepared for the consideration of the Administrative Conference of the United States. The views expressed are those of the author and do not necessarily reflect those of the members of the Conference or its committees. In undertaking this research, I benefitted from the insights from many individuals with expertise in U.S. regulatory policy, including: Chandana Achanta, Don Arbuckle, Tim Bober, Emily Bremer, Carol Browner, Cary Coglianese, Dan Cohen, Bridget Dooling, Susan Dudley, Neil Eisner, Don Elliott, Art Fraas, Russ Frisby, Lisa Heinzerling, Randy Lutter, Michael Fitzpatrick, Gary Gensler, Michael Greenstone, Charles Griffiths, Gretchen Jacobs, Elaine Kamarck, Sally Katzen, Demetrios Kouzoukas, Jim Laity, Jeff Lubbers, Dom Mancini, Charlie Marcesa, Al McGartland, Shawne McGibbon, Dick Morgenstern, John Morrall, Alan Morrison, Jennifer Nash, Nicole Owens, Bill Pine, Connor Raso, Michael Ravnitzky, Lisa Robinson, Jason Schwartz, Jasmeet Seehra, Howard Shelanski, Peter Strauss, Rich Theroux, Jim Tozzi, Paul Verkuil, Kip Viscusi, Jonathan Wiener, Matt Wiener, and participants at the August 2014 Council of Independent Regulatory Agencies meeting hosted by ACUS, participants at multiple meetings of the ACUS Regulation Committee, and participants at the Harvard Kennedy School Regulatory Policy Seminar. I did not provide a common set of questions in my conversations with these individuals. Reeve Bull provided very thoughtful comments and edits on several drafts that significantly improved this report. Lizzie Burns provided excellent research assistance. All omissions and commissions are the sole responsibility of the author.
mailto:[email protected]://www.hks.harvard.edu/fs/jaldy/
3
Contents Executive Summary ....................................................................................................................................... 4
Introduction .................................................................................................................................................. 8
Academic Evidence on Retrospective Review ............................................................................................ 17
Retrospective Analysis: Statistical Methods for Causal Inference .......................................................... 17
Lessons from Ex Post Validation ............................................................................................................. 22
The Role of Academic Research Centers in Retrospective Review ......................................................... 25
Retrospective Reviews under Previous Administrations ............................................................................ 27
Past Retrospective Reviews .................................................................................................................... 27
Lessons from Past Retrospective Reviews .............................................................................................. 34
Proposals for Reform of Retrospective Review .......................................................................................... 37
Regulatory PAYGO................................................................................................................................... 37
Regulatory Review Commissions ............................................................................................................ 39
Creation of Independent Regulatory Review Authorities ....................................................................... 40
Obama Administration Retrospective Review ............................................................................................ 42
President Obama’s Executive Orders...................................................................................................... 42
Lessons Learned from Obama Administration Retrospective Review .................................................... 45
Recommendations ...................................................................................................................................... 64
Retrospective Review Guidelines ............................................................................................................ 64
Integrating Retrospective Review into New Regulations ....................................................................... 66
Independent Review ............................................................................................................................... 67
Regulatory Coordination ......................................................................................................................... 67
Cumulative Regulatory Burden ............................................................................................................... 69
Public Participation ................................................................................................................................. 70
Resources ................................................................................................................................................ 71
References .................................................................................................................................................. 72
Tables .......................................................................................................................................................... 89
Appendix Tables ........................................................................................................................................ 106
4
Executive Summary
A well-functioning Federal regulatory program makes the American people better off by
promoting innovation; encouraging competition; protecting the air we breathe, water we drink, and
food we eat; and improving the safety of our workplaces and the goods we buy. Determining if we are
getting the most out of our regulatory program requires rigorous analysis. Such analysis can address
fundamental questions about regulatory policy: Do government regulations deliver on societal
objectives (such as those established by Congress)? Do regulations maximize net social benefits? Are
regulations enabling society to achieve our goals at the lowest possible cost?
Despite a long track record of prospective analysis of proposed regulations that can address
these questions, the Federal government has a mixed track record on retrospective review of existing
rules. Every administration dating back to the Carter Administration in 1978 has implemented some
form of regulatory look-back. In addition, agencies undertake retrospective review under their own
statutory authorities and the Regulatory Flexibility Act. The ad hoc nature of the Presidential-mandated
reviews, the apparent need for every administration to implement such a retrospective review, and the
heterogeneity in approaches to retrospective review by agencies suggest that efforts to enhance and
institutionalize retrospective review are merited.
The Administrative Conference of the United States has requested this assessment of
retrospective review of existing regulations. In particular, this assessment evaluates the practice of the
Obama Administration’s retrospective review, and places it in the context of the academic literature and
past administrations’ efforts at retrospective review. In addition, this assessment identifies best
practices among the agencies, describes key lessons learned from the ongoing and past retrospective
review efforts, and makes recommendations for way to improve retrospective review. There are two
general types of objectives for regulatory policy, one type advanced by Congress and the other
5
advanced by the White House. Through legislation, Congress establishes goals for specific regulatory
agencies. Through executive orders, Presidents (since President Reagan) have established a net social
benefits goal for Federal regulatory policy. As a result, I have framed my assessment and my
recommendations in terms of those actions that could improve the efficacy – i.e., attaining
Congressionally-established goals – and enable an increase in the net social benefits of Federal
regulatory policy. Of course, there are statutory policy objectives that may not be consistent with
maximizing net social benefits, so I also address cost-effectiveness, i.e., minimizing the burden of
attaining goals.
In implementing President Obama’s executive orders on retrospective review of regulations,
agencies identified tens of billions of dollars of cost savings and tens of millions of hours of reduced
paperwork and reporting requirements through modifications of existing regulations. Within two years
of issuing their final plans for retrospective review, executive branch agencies had completed more than
one-third of the 650+ planned reviews, with more than 90 percent of them resulting in amendments to
the Code of Federal Regulations. A few examples illustrate the potential for retrospective review to
deliver substantial benefits to society. The Department of Labor modified its chemical hazard labeling
requirements so that they would conform to the international standard, thereby reducing costs to U.S.
manufacturers – especially those looking to export to foreign markets – by about $2.5 billion over five
years. The Department of Health and Human Services streamlined reporting requirements and
burdensome regulatory obligations on hospitals that will deliver $5 billion in cost savings over five years.
The Environmental Protection Agency, recognizing regulatory overlap with the Department of
Agriculture, removed requirements on the dairy industry that will deliver about $650 million in cost
savings over five years.
The Obama Administration retrospective review effort focused on the importance of developing
a culture of retrospective review. The plan development process, regular and continuing engagement
6
with the public, and semi-annual reporting of the status of implementing retrospective review plans
serve to promote this culture. Nonetheless, a review of recent economically significant rules show that
no more than about 10 percent are the result of retrospective review of existing rules, and none include
plans for conducting a future retrospective analysis.1 In evaluating retrospective review efforts, a key
challenge is that the counterfactual – what would have happened in the absence of the Obama
Administration executive orders – is unknown and unknowable. Nonetheless, I draw lessons learned
based on what constitutes best practices for analysis, evaluation and review from agency practices.
Based on my evaluation of the retrospective review programs under the Obama Administration
and previous administrations, I recommend the following for improving and institutionalizing
retrospective review:
• Retrospective Review Guidelines: The Office of Management and Budget should work with
regulatory agencies to develop guidelines for retrospective review. This guidance should inform
agency efforts in: (a) designing a process for identifying and prioritizing rules for review; (b)
developing plans for retrospective analysis in the design of new regulations; and (c) conducting
ex post analysis; undertaking retrospective analysis.
• Integrating Retrospective Review into New Regulations: Well-designed regulations should
enable retrospective analysis to identify the impacts caused by the implementation of the
regulation. For a given select, economically significant rule, agencies should present in the rule’s
preamble a framework for reassessing the regulation at a later date. Agencies should describe
the methods that they intend to employ to evaluate the efficacy of and impacts caused by the
regulation, using data-driven experimental or quasi-experimental designs where appropriate.
Research design teams – drawing from experts in statistical, program, and policy evaluation
1 “Economically significant” rules are defined under Executive Order 12866, and typically have an annual effect on the economy of $100 million or more. “Major rules,” under the Congressional Review Act, have a very similar definition, including this $100 million economic effect threshold.
7
offices across the government – should work with regulatory agencies to develop rigorous
research designs in the development and implementation of new rules.
• Independent Review: Agencies should consider assigning the primary responsibility for
conducting retrospective review to a set of officials other than those responsible for producing
or enforcing the regulation.
• Regulatory Coordination: The Office of Management and Budget and regulatory agencies should
promote efforts to facilitate better regulatory coordination, to reduce overlap among existing
regulatory programs, and facilitate coordination with international trading partners’ regulatory
programs.
• Cumulative Regulatory Burden: The Office of Management and Budget and the Council of
Economic Advisers should coordinate an interagency process to develop options for estimating
the cumulative burden of the Federal regulatory program. These options should then be subject
to public comment and tasked to a National Research Council committee for review and
evaluation.
• Public Participation: Agencies should continue to actively engage the public on retrospective
review and explore all avenues for soliciting data and analysis by stakeholders and by providing
data to stakeholders to encourage their independent replication of retrospective analysis.
• Resources: Enhancing and institutionalizing retrospective review will require additional
resources, and the Administration and Congress should explore ways to provide resources for
doing so.
8
Introduction
If markets work, then government regulation is unnecessary. Indeed, government intervention
in well-functioning markets will likely make society worse off by imposing costs that exceed their
benefits. If markets do not work well, as a result of market power, asymmetric information, public
goods, or externalities, then government regulation has the potential to remedy the market failure and
make people in society better off. This is not guaranteed. A poorly designed regulation, even if
motivated by a market failure, could result in costs in excess of the benefits and would exacerbate the
welfare losses. A well-designed regulation, however, can improve the welfare of affected people and
attempt to deliver what the market would if it were not suffering from the market failure.
How does the government identify the need for regulation and discern among various
regulatory options to determine how best to improve the welfare of the American people? For more
than three decades, the Federal government has employed a process of assessing the regulatory
impacts of proposed regulations. Such analyses produce estimates of expected benefits and costs that
can address a fundamental question of regulatory policy: does the regulation increase societal welfare?
Such analyses also provide the basis for determining the extent to which the regulatory action advances
the statutorily-established objective. The use of ex ante analysis has shed light on lower cost ways of
achieving a societal goal. This analysis has also driven research agendas so that agencies can better
understand the impacts – costs on regulated firms and benefits to various populations – of their
regulatory actions. It has motivated efforts to reach out to regulated industries to help identify lower
cost ways of correcting market failures. It has helped various stakeholders identify and advocate for
priorities in agencies’ regulatory programs.
This is not to say that all regulations maximize net social benefits. Agencies promulgate
regulations subject to their statutory authority, which, in many cases places constraints on how an
9
agency can design its regulation. A statute may prohibit an explicit consideration of benefits and costs in
the design of regulations. Alternatively, a statute may prescribe the regulatory intervention and provide
little discretion to the regulator. Some regulatory agencies, which are not governed by executive orders
on regulatory policy, may employ non-economic decision criteria in their rulemakings. As a result, some
regulations may fail to deliver societal benefits cost-effectively, but nonetheless result in net social
benefits. There may also be cases in which the societal costs exceed societal benefits. In this case,
analysis of these impacts may not provide the legal basis for an agency to defy the mandate from
Congress in the pertinent authorizing legislation, but it does serve to highlight opportunities for future
legislative reform of that authority.
This process of assessing the regulatory impacts of proposed regulations, with heightened
scrutiny for those that would have significant economic impact, has established a culture of prospective
analysis. There is, however, less activity, a mixed track record, and fewer resources directed to ex post
assessment of Federal regulations. Every administration since the Carter Administration has
implemented some kind of retrospective review of regulations, yet there is little doubt that most
agencies dedicated less attention to retrospective review than prospective review. Most economically
significant regulations, while subject to rigorous ex ante analysis, are not designed to produce the data
and enable causal inference of the impacts of the regulation in practice. Some agencies employ fairly
systematic approaches to reviewing existing rules, either as a result of the need to periodically update
regulations under their statutory authority or under the Regulatory Flexibility Act. Other agencies
employ less formal approaches that may reflect stakeholder demand – i.e., public comments and
complaints – as opposed to analysis.
President Obama has issued several executive orders that require retrospective review and
attempt to institutionalize such reviews in agencies’ regulatory programs. Given the value that such
analysis can have in improving the Federal regulatory program, the Administrative Conference of the
10
United States has requested this assessment of President Obama’s regulatory look-back initiatives. In
particular, the Administrative Conference of the United States has charged this project to:
• examine the various agency approaches to retrospective reviews and to identify potential best
practices for review planning, including how to identify priorities and determine which
regulatory programs are good candidates for retrospective review;
• identify characteristics of successful reviews, including guidance on effective analytical
processes, the use of public input, and the development of retrospective plans in the
development of new rules; and,
• suggest recommendations for improving retrospective review.
This is not the first time that the Administrative Conference of the United States has explored
the issue of retrospective review. Shapiro (1995) conducted an assessment of retrospective review,
drawing in part from an American Bar Association survey of regulators (see Eisner et al. 1996 for details
of this survey and the conclusions they draw from it). The Administrative Conference of the United
States adopted recommendation 95-3 based on this report, which called for the following:
• Retrospective review should inform potential changes to existing rules: “All agencies should
develop processes for systematic review of existing regulations to determine whether such
regulations should be retained, modified, or revoked.”
• One size does not fit all: “Systemic review processes should be tailored to meet the need of each
agency.”
• Priority setting: “Agencies should establish priorities for which regulations are reviewed…. In
setting such priorities, the following should be considered:
• whether the purpose, impact, and effectiveness of the regulations have been impaired by
changes in conditions;
11
• whether the public or regulated community views modification or revocation of the
regulations as important;
• whether the regulatory function could be accomplished by the private sector or another
level of government more effectively and at a lower cost; and
• whether the regulations overlap or are inconsistent with regulations of the same or
another agency.”
• Public input: “Agencies should provide adequate opportunity for public involvement in both the
priority-setting and review processes.”
• Implementation: “Agencies should provide adequate resources;” “where appropriate, should
engage in risk assessment and cost-benefit analysis of specific regulations”; and consider
frequency of review and categories of regulations to be reviewed.
As the discussion below of the various retrospective reviews since 1978 and the current practice
by some agencies under the Obama Executive Orders illustrates, a number of these recommendations
have been and continue to be followed today. Nonetheless, the lessons that can be drawn from the
experiences with retrospective review and the improvement in data collection and analytic methods
suggest that additional steps can be taken to improve the information in retrospective review and build
a culture of retrospective review such that it can generally enhance the quality of Federal regulatory
policy.
In undertaking this project, I have evaluated the retrospective review plans and progress reports
of the following agencies: Departments of Agriculture, Energy, Health and Human Services, Homeland
Security, Justice, Labor, and Transportation, the Environmental Protection Agency, the Consumer
Financial Protection Bureau, the Commodity Futures Trading Commission, the Federal Reserve System,
the Nuclear Regulatory Commission, and the Securities and Exchange Commission. These agencies
represent the universe of Federal regulatory authorities that each promulgated more than one
12
economically significant rulemaking over the FY2003-FY2012 period according to the Office of
Management and Budget (2013a, Table 1-1 and Table C-2). This sample includes a mixture of both
executive branch and independent regulatory agencies.
I have also reviewed two strands of important academic literature: one on the statistical
methods for rigorous estimation of ex post impacts of regulations and one on the insights drawn from
meta-analyses of studies that undertake ex post validation of ex ante benefits and costs estimates. I
synthesize key insights from the academic literatures on ex post regulatory review and rule-specific ex
post empirical analyses. For example, Harrington et al. (2000), Office of Management and Budget
(2005), Harrington (2006), Harrington et al. (2009), and Council of Economic Advisers (2012) provide
examples of ex post validation of ex ante benefit and cost estimates of Federal regulations. In addition,
several studies have evaluated the cost-effectiveness of Federal regulations intended to reduce
mortality risk across an array of agencies (e.g., Morrall 1986; Tengs et al. 1995; Hahn et al. 2000) and
illustrated the variation in key benefits assumptions across agencies (e.g., Viscusi and Aldy 2003). Aldy
and Viscusi (2014) describe how uncertainty can affect the efficiency and efficacy of regulatory
interventions that further demonstrates the value of ex post examination.
I also review past practice with retrospective review, including under specific agencies’ statutory
authorities and the Regulatory Flexibility Act, as well as Presidential mandates in the Carter, Reagan,
Bush I, Clinton, and Bush II administrations. I explore the commonalities and the differences in these
approaches and attempt to ascertain why retrospective review is still often described as “ad hoc”
despite this multitude of processes.
In this evaluation of retrospective review, my analysis is predicated on this question: how can
the process of retrospective review be improved to yield a regulatory program that delivers on
statutorily-established societal goals and maximizes net social benefits of government interventions in
the economy? This clearly reflects the bipartisan objective in the Reagan and Clinton visions of
13
regulatory policy (the latter of which still governs today through Executive Order 12866). This also
reflects statutory authorities that explicitly direct agencies to consider the benefits and costs of their
regulations. For example, the Department of Energy shall “determine whether the benefits of the
standard exceeds its burdens” in establishing minimum energy efficiency standards (42 USC 6295). In
regulating chemicals under the Toxic Substances Control Act, the Environmental Protection Agency
“shall consider… the effects of such substance on health… [and] on the environment, … the benefits of
such substance… for various uses, and the reasonably ascertainable economic consequences of the rule”
(15 USC 2605). The 1936 Commodity Exchange Act specifies that costs and benefits of proposed
regulations by the Commodity Futures Trade Commission shall be evaluated (7 USC 19). As a result, I
look for ways to identify the rules that could most benefit from evaluation and potential reform.
Given that Congress occasionally establishes regulatory objectives that preclude consideration
of benefits and costs in rulemakings and sometimes prescribes constraints on regulators through its
statutory authorities, it is important to consider how improvements in the process of retrospective
review promote the attainment of goals identified in authorizing legislation? Can we improve the
efficacy of the Federal regulatory program? If so, can we also do so in a cost-effective manner? Some
agencies operate under mandates that do not necessarily square well with a “maximize net social
benefits” standard. For example, several agencies operate under statutory authorities focused on
protecting public and safety, including the Occupational Safety and Health Administration (29 USC 655),
the Environmental Protection Agency under the Clean Air Act (42 USC 7409), and the Nuclear Regulatory
Commission (42 USC 2113, 2167, 2210e, etc.). Nonetheless it is important to understand how regulatory
performance stacks up against the specified objectives established by Congress. It is also important to
explore potentially novel ways to achieve such goals, especially given the insights gained in practice
through performance-based regulation, market-based approaches to regulation, and behavioral
economics.
14
To be clear, a focus on maximizing net social benefits is not the deregulatory bias that is evident
in some rhetoric around retrospective review (which I discuss below in the history of President-initiated
retrospective reviews). There are clearly rules that, as a result of new information and analysis, are
insufficiently stringent given their costs and benefits. For example, the sulfur dioxide cap-and-trade
program established under Title IV of the 1990 Clean Air Act Amendments was motivated by concern
about acid rain, which could result in acidification of streams and lakes and forest die-off. In the years
after the implementation of this program, epidemiological research presented important evidence on
how reducing sulfur dioxide emissions, and their associated fine particulates, delivers quite substantial
human health benefits. As a result, the ecological benefits, which drove the initial rule, may be less than
the compliance costs, but the human health benefits – which were not considered by Congress or the
Environmental Protection Agency in the design of the program – are about 100 times the compliance
costs (Schmalensee and Stavins 2013). A retrospective review of this rule would suggest that the
Environmental Protection Agency should make the case to Congress for flexibility in setting the
emissions cap in this program so that it can deliver more human health benefits that significantly exceed
the costs.2
To preview some of the lessons drawn from the assessment of the regulatory experience and
the relevant scholarship, a well-designed system of retrospective review should strive for the following.
Given scarce resources, reviews should be implemented in order to produce the greatest net social
benefits. This can require a consideration of those rules that operate in environments in which pertinent
circumstances have changed – such as an evolution in technology or a change in the statutory authority.
This can reflect an understanding of new estimates of benefits and costs. It could also exploit the
opportunity that some retrospective evaluations can provide insights that could benefit the review as
2 Indeed, there were a number of legislative proposals in the 2000s to revise and update Title IV of the Clean Air Act. For example, the Bush Administration introduced a “Clear Skies Proposal” in 2002 and Senator Carper introduced several so-called “3P” bills that would modify regulation of power plant emissions of sulfur dioxide, nitrogen oxides, and mercury.
15
well as the future design of various other regulations (a kind of knowledge spillover). It may also identify
the gaps in the regulatory program, i.e., opportunities for new regulation to address an emerging
market failure.
Maximizing net social benefits will necessitate a rigorous evaluation of the ex post impacts of
the regulation. Successfully doing so will often involve planning for such an evaluation in the
development and implementation of the rule. It will also likely benefit from tapping resources
throughout the Federal government, including experts in statistical agencies and policy and program
evaluation shops in various departments.
The operation of a retrospective review process can benefit from active public engagement to
assist in identifying rules as well as data and analysis to facilitate their evaluation. It is important to
recognize however, that public engagement should not serve as a substitute for rigorous evaluation.
Sometimes the regulated community may have loud complaints about a given rule, but that simply
reflects their take on the costs side of the ledger. Other times the regulated community may be silent
about a given rule, because they have already incurred the investment costs for compliance and the rule
now represents a barrier to entry that mitigates the competition they face in markets. Thus, public
comment could always benefit from being supplemented by analysis. Agencies may also consider ways
to lower the cost of public engagement, including various uses of social media, online dissemination of
data analysis, communication through plain language, and other forms of outreach that can enable
participation by more than just the best-funded stakeholders.
Finally, retrospective review can enhance social welfare by taking a bigger picture perspective
than is typically the case in the development of new regulations. Given concern about potential
regulatory duplication as well as the cumulative impact of regulations on firms, efforts should be
undertaken to promote coordination among agencies and comprehensive assessment of cumulative
impacts of regulatory programs.
16
The next section of this report presents the assessment of the scholarship on retrospective
review and ex post analysis of regulatory impacts. The third section describes and evaluates past
administrations’ retrospective review efforts as well as various statutory requirements for retrospective
reviews. The fourth section briefly reviews ideas for legislative reforms for retrospective review. The
fifth section provides an examination of the Obama Administration executive orders on retrospective
review and the agency practices in implementing these executive orders and draws lessons to inform
future retrospective review efforts. The final section presents recommendations for improving
retrospective review of regulations.
17
Academic Evidence on Retrospective Review
A very rich literature has evolved over the past several decades focused on evaluating ex post
the impacts of various regulatory interventions. These empirical analyses have produced estimates of
the realized benefits, costs, efficacy, cost-effectiveness, and, in some cases, unintended consequences
of Federal regulations. Researchers have employed an array of statistical techniques that provide for a
more robust understanding of the phenomena associated with these rules that can complement the
tools used by regulatory agencies in their ex ante analyses and can inform subsequent rounds of
regulatory actions. This section reviews the methods employed in retrospective analysis, then draws
some lessons from ex post validation exercises, and closes with a discussion of the role played by
academic research centers in retrospective review of regulatory policy.
Retrospective Analysis: Statistical Methods for Causal Inference
The most straightforward way to investigate the impact of a given regulation or other
government intervention would be through a randomized control trial. Under such an approach, some
firms would be randomly assigned “regulated” and other firms would be randomly assigned “not
regulated,” and the analyst could compare the differences in outcomes and attribute this difference to
the impact of the rule. The objective is to attempt to identify a “treatment” group and a “control” group,
like a laboratory experiment, and to use the control group to serve as an effective counterfactual.
Such a randomized control trial approach is feasible for testing information disclosure
instruments. For example, Lacko and Pappalardo (2010) investigate how well homebuyers understand
residential mortgage terms through status quo information disclosure – the Department of Housing and
Urban Development’s Good Faith Estimate of Settlement Costs form (GFE) and the Federal Reserve
18
Board’s Truth in Lending Statement (TILA) – and an alternative prototype. The researchers developed
the prototype in an effort to improve borrower understanding of residential mortgages. The researchers
identified a population of recent mortgage borrowers and randomly assigned each individual this
population a disclosure form – either the GFE/TILA or the prototype – that detailed a 30-year fixed rate
loan for a hypothetical residential property. In effect, those individuals receiving the prototype
disclosure are the control group for comparison with the treatment group, those who received the
status quo GFE/TILE disclosure.3 After reviewing the disclosure forms, study participants completed a
comprehension questionnaire. Lacko and Pappalardo found dramatically higher rates of comprehension
with the prototype, suggesting that improving the design of mandatory disclosure forms could benefit
consumers.4
It may be quite challenging to implement randomized control trials in practice in other
regulatory contexts, given political, legal, and, in some cases, ethical reasons. Nonetheless, statutory
authorities may result in regulatory implementation that could facilitate the identification of a
“treatment” group and a “control” group that would serve as the basis for rigorous statistical analysis.
For example, a statute may call for regulatory implementation in phases and thus those covered by a
later phase of implementation could serve as the control to those covered by the initial phase of
implementation (the treatment). A statute may permit pilot programs and a well-designed pilot could
result in treatment and control groups. Some statutes authorized regulators to provide guidance to the
states, which are responsible for implementation. It is possible that the heterogeneity in state efforts
under these regulations could provide a means for statistically evaluating alternative implementation
strategies. Some rules may establish objectively measured standards, above which may trigger
3 Given the nature of this experiment, one could also frame it as the GFE/TILA recipients are the control group and the prototype disclosure recipients are the treatment group. The results are the same regardless of the initial framing. 4 Sunstein (2011) notes the importance of such testing of alternative presentations of information before promulgating rules on mandatory disclosure.
19
regulation (treatment) and below which does not (control). The key issue is that regulators do not
necessarily pick who is to be regulated and who is not to be regulated in order to permit ex post
statistical analysis. Instead, it is the nature of the statutory authority that often will do so.
As a result, analysts employ a variety of so-called quasi-experimental approaches, such as
difference-in-differences, propensity-score matching, instrumental variables, and regression
discontinuity statistical techniques (explained in further detail below).5 Just as with a randomized
control trial, the objective in all of these techniques is to attempt to identify a “treatment” group and a
“control” group. Statistically, this can be quite challenging since firms, or individuals, or regions may not
be randomly assigned to one group or another, and in fact they can often change their behavior to
select into one group or another. For example, if a clean air regulation only applies to power plants
larger than a specified generating capacity, then a utility may build a power plant just below this
threshold to avoid the regulation. Likewise, if a labor regulation only applies to firms with employees in
excess of a minimum threshold, then firms may manage payrolls to stay below this minimum. Moreover,
those subject to treatment may just be fundamentally different than those identified as control, which is
not necessarily unexpected since those are the ones targeted for regulation.
Consider a few examples of these research design techniques in the existing regulatory contexts.
Given its name, difference-in-differences approaches focus on two differences: the difference before
and after the timing of a regulatory intervention and the difference between the treatment (regulated)
and the control (unregulated) groups. The first difference attempts to control for possible time trends
that could contaminate the estimation of the effect of regulation (e.g., technological innovation may,
without the impact of government intervention, result in cleaner-burning fuels in vehicles) and would be
common across treatment and controls. After accounting for this first difference, the second difference
5 Greenstone (2009), Coglianese (2013d, 2012b), and Coglianese and Bennear (2005) address the need for rigorous research design in the development of regulations in further detail. See DiNardo and Lee (2011) for a more technical review of statistical policy evaluation tools.
20
then attempts to isolate the impact of the regulation on outcomes in the treatment. A number of
studies have employed versions of this approach in investigating the impacts of the national ambient air
quality standards, in which the designation of a county as non-attainment represents the treatment, and
those left in the attainment category are viewed as control. These papers have shown the impacts of
non-attainment designations on costs, employment, and emissions under the Clean Air Act (Henderson
1996; Becker and Henderson 2000; Greenstone 2002).
Using such methods, Greenstone et al. (2006) found that increasing information disclosure of
equities improved returns to shareholders and firm performance under the implementation of the 1964
Securities Acts Amendments. In their study, they took advantage of the fact that prior to these
amendments, only exchange-traded firms were required to disclose financial performance information.
Under the amendments, these information disclosure requirements were extended to some, but not all,
over-the-counter traded public firms. These over-the-counter firms, whose disclosure requirements
changed, were the treatment group in the analysis, and the researchers employed two distinct control
groups – the exchange-traded firms that had the information disclosure requirements before and after
the amendments (and whose regulatory status did not change) and the small over-the-counter firms
who were not required to disclosure financial information through the amendments (and whose
regulatory status also did not change).
Propensity-score matching employs a similar approach to difference-in-differences, but uses
statistical methods to identify control observations that appear, with the exception of treatment status,
to be similar to treatment observations. This approach effectively down weights observations on firms
or regions that are just fundamentally different – as evident in pre-regulation characteristics – than
those impacted by a government regulation. The researcher conducts statistical analysis to identify
“best matches” for each treatment observation among the pool of potential control observations based
on pre-regulation data. Abadie et al. (2010) demonstrate a new variation on this approach in an
21
investigation of tobacco regulations in California. Their synthetic control approach effectively creates a
control group that is a composite of all potential control population members. The composite is
constructed such that it best matches the treatment group pre-treatment – i.e., showing tobacco
consumption in California equal to tobacco consumption in the constructed synthetic control region
before the implementation of the state’s tobacco control program. This constructed synthetic control
region is based on the weighted average of other state’s tobacco consumption that best fits the pre-
treatment California data. Then the impact of the state tobacco control program in California is
estimated based on the differences between California tobacco consumption and the synthetic control
group post-regulation.
Regression discontinuity approaches take advantage of assignment of a discrete threshold that
determines whether a firm or a region is assigned to regulatory treatment or control. Comparing
observations of outcomes for those firms (or regions) just above and just below the threshold, the
analyst can generate an estimate of the so-called average treatment effect of the regulation. For
example, Benner and Olmstead (2008) employed this approach to investigate the impact of information
disclosure on drinking water quality. Under the 1996 Safe Drinking Water Act Amendments, drinking
water suppliers serving a population above a specified threshold were required to produce information
summaries on drinking water violations and send this information out to all customers, while those
below the threshold only had to post such information in a public space (e.g., a town hall or library).
They found that drinking water violations fell for those subject to the greater level of transparency after
the implementation of the information disclosure requirement. Likewise, Berry and Lee (2007) employ
regression discontinuity methods to investigate the impacts of the Community Reinvestment Act. They
took advantage of the fact that Community Reinvestment Act rules apply to neighborhoods with
incomes below 80 percent of metropolitan statistical area median income, and compared lending
22
activity in neighborhoods just above and just below this threshold. They found no impact of the act’s
regulations on loan rejection rates near this threshold.
Common to all of these techniques is the motivation of structuring the statistical model to
enable causal inference, not simply correlation or association. Indeed, failing to do so could yield quite
misleading results. For example, if an agency implemented a regulation in 2009, and then a number of
regulated firms had closed by the end of 2010, one might claim that this reflected the burden of the
regulation. Yet, one could not reject the counterclaim that the fall in demand in the Great Recession
may have resulted in these firm closures. Only through a careful research design can an analyst discern
causation from association. It is important to note a caveat in much of the empirical scholarship to date.
Any given research paper may not map one-to-one to a specific regulation. Instead, the researchers may
be using a measure of total regulatory compliance costs, or total air quality regulatory obligations, given
their data availability. Researchers may not be able to parse out the impacts of air quality regulation A
from air quality regulation B from air quality regulation C on a given industry or sector. This highlights
the need for careful research design ex ante – i.e., when rules are proposed – to ensure that the data
and the implementation of the regulation can permit causal inference in a future, ex post analysis.
Lessons from Ex Post Validation
Harrington et al. (2000) conducted an ex post validation exercise by comparing approximately
twenty ex post assessments of the benefits and costs of regulations with their ex ante estimates. They
found evidence of overestimates of benefits and costs, but these biases disappeared on a per unit basis.
The higher ex ante benefit and cost estimates reflected assumptions of 100 percent compliance. With
less than full regulatory compliance in practice, realized benefits and costs were lower. The authors
found that the ex post per unit impacts validated the ex ante estimates. This suggests that accounting
23
for compliance behavior may be important in assessing the ex post validity of ex ante estimates. This
study spurred subsequent work focused on a larger set of regulations. The Office of Management and
Budget (2005) conducted a validation exercise of the benefits and costs of 47 regulations issued over
1975-1996 as a part of its annual report to Congress on the benefits and costs of Federal regulations.
The exercise compared the ex ante estimated benefits and costs published in the regulatory impact
analyses for these rules and ex post estimates published by academics and government agencies. In this
assessment, the Office of Management and Budget contrasted the ex ante and ex post estimates of the
physical quantities of the primary benefits category (e.g., tons of emission reduction) and the monetized
measure of costs. The Office of Management and Budget (2005) notes that the purpose of this exercise
was “to summarize the findings from this validation literature, identify possible explanations for
inaccuracies that are identified, and discuss possible ways that the validity of ex ante estimates of
benefits and costs can be improved” (p. 42). Thus, in contrast to past presidents tasking agencies to
undertake retrospective review and select rules to be revised or eliminated, this effort had a more
modest goal of providing insights on ways to improve the conduct of regulatory impact analysis.
The Office of Management and Budget concluded that both ex ante benefits and costs are
overestimated by regulatory agencies, although with more overestimation of benefits and a bias toward
overestimating benefit-cost ratios. Harrington (2006) evaluated this scoring by the Office of
Management and Budget and made a number of modifications of the Office of Management and Budget
sample. In his review of 60 case study regulations, Harrington found that agencies were more likely to
underestimate benefit-cost ratios than overestimate them. A common conclusion drawn in these
analyses is that there are too few ex post studies for such validation exercises to inform regulatory
review.
The Harrington et al. (2000) study also found that for rules that employed market-based
implementation strategies, there was clearer evidence that the prospective regulatory impact analyses
24
overestimated the costs. This may reflect the failure of imagination intrinsic to such a prospective task.
One of the primary motivations for market-based approaches is to provide the flexibility and discretion
to regulated entities to be creative and come up with the lowest cost way of realizing the societal goal in
the regulation. With this freedom, regulated entities have the profit-incentive to seek out and exploit
the lowest-cost compliance strategies, some of which may have been beyond the scope of consideration
by the regulator. Ex post analysis provides the regulator with this understanding that can inform the
choice and design of regulatory instrument in future regulations.
A number of papers have also shown empirically how the use of market-based approaches to
regulation, such as emissions cap-and-trade, can result in lower compliance costs than conventional
command-and-control regulatory approaches. For example, Kerr and Newell (2003) estimated
substantial cost-savings associated with allowing refineries to trade lead credits during the phase-out of
leaded gasoline. Carlson et al. (2000) estimated that the sulfur dioxide cap-and-trade program likely
resulted in about half of the compliance costs of an alternative performance-based approach to sulfur
pollution. Likewise Ellerman et al. (2000) undertook an extensive variety of empirical analyses of the
sulfur dioxide cap-and-trade program and found important cost-savings. The key result of this
scholarship, and others in the cap-and-trade literature, is that market-based approaches can deliver
lower-cost compliance than traditional regulatory approaches.
Ex post analyses may also highlight the unexpected or unintended in regulatory implementation.
For example, very few ex ante analyses in the environmental, health, and safety context consider the
potential for pre-existing market failures – such as market power – to impact the societal costs
associated with regulatory compliance. Ryan (2011) develops a structural industrial organization model
and shows how accounting for the dynamics of firm entry and the investment costs associated with air
quality regulations in the Portland cement industry, the estimated costs of compliance are substantially
greater in an industry characterized by imperfect competition. In an early empirical assessment of
25
automobile safety regulations, Peltzman (1975) found that mandating various types of safety
equipment, such as seat belts, resulted in a behavioral response among drivers (moral hazard). While
the safety equipment reduced premature mortality among drivers, their riskier driving behavior resulted
in an increase in pedestrian mortalities. Gruenspecht (1982) found that imposing costly air quality
controls on vehicles increased their sales price, which slowed the turnover of automobile ownership
thereby leaving more, high-polluting old cars on the road and increased air pollution in the first few
years of the regulation.
In some cases, ex post analysis can resolve uncertainties in the underlying risk assessments that
motivate regulatory interventions. Kolp and Viscusi (1986) and Viscusi (1985) point out analytic errors as
well as substantial uncertainties in health risks – exposure and magnitude of impacts – in their
retrospective review of the 1978 Occupational Safety and Health Administration cotton dust standard.
Thompson et al. (2002) point out that the ex post benefits estimates of air bag regulations differed from
ex ante estimates due to ex ante overestimation of air bag effectiveness, overestimation of baseline
fatality and injury rates, and underestimation of the rate of seatbelt usage.
The Role of Academic Research Centers in Retrospective Review
Academics can play and indeed have played a very active role in the design and implementation
of retrospective review. For example, the George Washington University Regulatory Studies Center,6 the
Mercatus Center at George Mason University,7 the New York University School of Law Institute for Policy
Integrity,8 and the Penn Program on Regulation at the University of Pennsylvania Law School9 have all
participated regularly through scholarship and public comments on retrospective review. The Institute
6 http://regulatorystudies.columbian.gwu.edu/. 7 http://mercatus.org/. 8 http://policyintegrity.org/. 9 https://www.law.upenn.edu/institutes/regulation/.
http://regulatorystudies.columbian.gwu.edu/http://mercatus.org/http://policyintegrity.org/https://www.law.upenn.edu/institutes/regulation/
26
for Policy Integrity (2011a-f) submitted comments on draft and final retrospective review plans. The
Penn Program on Regulation hosts a blog with a number of commentaries on the practice of
retrospective review (Coglianese 2011, 2012a, 2012c, 2013a, 2013c). The Regulatory Studies Center has
an ongoing process of submitting comments on proposed rules focused on how the design and
implementation of those rules will facilitate ex post analysis. The Mercatus Center hosts papers and
testimonies by a number of scholars on the topic of retrospective review. Developing a cadre of
independent experts – and in training the next generation of regulatory policy experts – at these
academic research centers can provide thoughtful takes on the performance of the Federal regulatory
program. In any case, they provide a rich set of resources outside of government on the operation of the
regulatory program generally and the implementation of retrospective review.
27
Retrospective Reviews under Previous Administrations
Past Retrospective Reviews
The systematic review of existing regulations across the executive branch dates back, in one
form or another, to the Carter Administration. In 1978, President Carter issued Executive Order 12044,
“Improving Government Regulations,” which created a “cost-effectiveness” standard for regulatory
policy, required regulatory analysis for significant regulations, and established the White House as
responsible for working with regulatory agencies in insuring implementation of the executive order
(Viscusi 1994). This order also required regulatory agencies to undertake periodic review of their existing
regulations and provided the following criteria for identifying rules for retrospective review:
• “the continued need for the regulation;
• the type and number of complaints or suggestions received;
• the burdens imposed on those directly or indirectly affected by the regulations;
• the need to simplify or clarify language;
• the need to eliminate overlapping and duplicative regulations; and
• the length of time since the regulation has been evaluated or the degree to which technology,
economic conditions or other factors have changed in the area affected by the regulation”
(section 4).
The Carter Executive Order provided agencies with 60 days to develop a draft report detailing
their process for developing and evaluating regulations as well as their criteria for choosing rules for
retrospective review. The agencies published their draft reports in the Federal Register and solicited
public comment on their content.
28
The Carter executive order created a new regulatory framework at a time of dramatic change in
Federal regulatory policy. A wide array of economic regulatory policies – from the setting of interstate
natural gas prices, to the regulation of railroad freight rates and routes, to the regulation of fares and
routes in civil aviation, among others – had come under scrutiny for imposing substantial burdens on
consumers, stifling innovation, and contributing to the bankruptcy of some firms (Joskow and Noll 1994;
Moore 2002; Winston 2007; Davis and Killian 2011). While much of the reform of economic regulation
occurred through various pieces of legislation, as opposed to administrative changes under existing
authorities, the wave of economic deregulation reflected a series of ex post assessments that the status
quo regulatory schemes did not work as intended. This occurred at a time, however, of substantial
growth in environmental, health, and safety regulation at the federal level. Creating a formal process of
regulatory review and of ex post review of existing regulations provides a rigorous basis for assessing
individual rules on their merits, as opposed to making regulatory decisions on ideological deregulatory
or proregulatory grounds.
The Carter executive order did not establish a benefit-cost standard for evaluating regulations.
Thus, the principles for retrospective review, such as the assessment of the burdens, lack potentially
important context. A rule that imposes $100 million of burdens, but $10 billion of monetized benefits,
may appear to any regulator as a fantastic success story, while a rule that imposes $10 million of
burdens to deliver $1 million of monetized benefits may be ripe for revision or rescission. The other
prime challenge in implementing review of existing regulations under this executive order was the
absence of standards for conducting ex ante analyses and requiring data collection to inform ex post
analysis. This simply reflects the fact that this executive order was the first to create a systematic
requirement for agencies to identify significant rules (without a specified standard in the executive
order) and undertake regulatory analysis.
29
The Carter Executive Order established an important precedent that was employed in the 1980
Regulatory Flexibility Act. Section 610 of this law requires periodic review of rules to determine if
existing regulation should be amended or rescinded to “minimize any significant economic impact of the
rules upon a substantial number of small entities.” The act employed virtually identical criteria as the
executive order for reviewing rules. The law also requires regulatory agencies to issue a plan for periodic
review that ensures that all economically significant rules are reviewed within ten years of their
promulgation.
The impact of retrospective review under the Regulatory Flexibility Act is, at best, mixed.
Regulatory agencies have employed various interpretations of the Act’s requirements (Copeland 2004).
For example, the Small Business Administration (2008) notes that the Environmental Protection Agency
and the Occupational Safety and Health Administration only review rules that were estimated, ex ante,
to have an economically significant impact on small entities. These agencies exclude rules that could
have a significant economic impact, which could only be ascertained by a review. As a result, the
effective review rate is low, which undermines the potential effectiveness of this requirement (See
2005). In 2008, the Small Business Administration (2008) published a “best practices” for Federal
agencies to promote more and higher quality Regulatory Flexibility Act retrospective reviews.
In 1981, President Reagan issued Executive Order 12291, “Federal Regulation,” which is
generally considered as the foundation for the current system of regulatory review and coordination.10
This executive order imposed five requirements on regulatory agencies in their development of new
regulations and review of existing regulations:
• “administrative decisions shall be based on adequate information concerning the need for and
consequences of proposed government action;
10 President Clinton’s Executive Order 12866, discussed below, makes a number of important departures from Executive Order 12291, but maintains a focus on regulatory impact analysis of rules with an annual effect on the economy of at least $100 million, continues the role of the Office of Management and Budget in coordinating review, and imposes a softer version of the benefit-cost standard.
30
• regulatory action shall not be undertaken unless the potential benefits to society for the
regulation outweigh the potential costs to society;
• regulatory objectives shall be chosen to maximize net benefits to society;
• among alternative approaches to any given regulatory objective, the alternative involving the
least net cost to society shall be chosen; and
• agencies shall set regulatory priorities with the aim of maximizing the aggregate net benefits to
society, taking into account the condition of the particular industries affected by regulations, the
condition of the national economy, and other regulatory actions contemplated for the future”
(section 2).
New regulatory actions that are expected to impose an annual impact of at least $100 million on
the U.S. economy triggered heightened levels of regulatory impact analysis and interagency regulatory
review. The executive order required agencies to publish twice annually a regulatory agenda that
includes a list of existing rules undergoing review by the regulatory agency. The order also calls on the
Office of Management and Budget and the Presidential Task Force on Regulatory Relief (led by Vice
President Bush) to identify existing rules that duplicate or conflict with other rules as well as those
existing rules that are “inconsistent… with the purposes of [the] order” (section 6) and to work through
the interagency process to eliminate the duplication or conflict.
Executive Order 12291 also tasked the Office of Management and Budget and the Presidential
Task Force on Regulatory Relief with developing procedures for estimating benefits and costs with the
intent of constructing a regulatory budget. As Viscusi (1994) notes, the idea of a regulatory budget had
some currency in the 1970s and 1980s, though it also had its critics (Viscusi 1983 provides a critique of
this approach to regulatory policy).11 Despite this provision in the executive order, the Reagan
Administration did not implement a regulatory budget, and this likely reflects both the technical and 11 This idea still draws interest today, as discussed below in the context of Senator Warner’s support for “regulatory PAYGO.”
31
coordination challenges within an agency as well as the political pushback the White House already
faced with the strict benefit-cost standard (Viscusi 1994).12
In 1985, President Reagan issued Executive Order 12498, “Regulatory Planning Process.” This
order built on Executive Order 12291 by requiring agencies to submit annual draft regulatory programs,
which the Office of Management and Budget compiled into an Administration-wide annual regulatory
program. The executive order called on each agency to identify the specific, significant regulatory
actions that would either rescind or revise existing regulations.
The George H.W. Bush Administration employed the two Reagan executive orders as its
framework for regulatory policy development and evaluation. In 1992, President Bush transmitted the
Memorandum on Reducing the Burden of Government Regulation to various heads of regulatory
agencies, which opens with “[a]s you know, excessive regulation and red tape have imposed an
enormous burden on our economy.” The Memorandum established a 90-day moratorium on new
regulations and required regulatory agencies to assess existing regulations and eliminate those that
impose “any unnecessary regulatory burden.” The Memorandum provided the following standards to
guide the agencies’ review of their existing regulations:
• “The expected benefits to society of any regulation should clearly outweigh the expected costs it
imposes on society.
• Regulations should be fashioned to maximize net benefits to society.
• To the maximum extent possible, regulatory agencies should set performance standards instead
of prescriptive command-and-control requirements, thereby allowing the regulated community
to achieve regulatory goals at the lowest possible cost.
12 The “strict” benefit-cost standard in Executive Order 12291 had an exception that it only applied to the extent permitted by law. As many scholars have noted over the years, some of the most economically significant regulations, such as the National Ambient Air Quality Standards promulgated by the Environmental Protection Agency, are authorized under statutory provisions that preclude a consideration of benefits and costs in their design (Arrow et al. 2000).
32
• Regulations should incorporate market mechanisms to the maximum extent possible.
• Regulations should provide clarity and certainty to the regulated community and should be
designed to avoid needless litigation” (Section 1).
This Memorandum represented a substantial change in the approach to retrospective review.
First, by coupling the review of existing regulations with a moratorium on new regulations, the
Memorandum effectively freed up staff resources to focus on retrospective review. In their survey of
sixteen regulatory agencies that reviewed existing rules under this Memorandum, Eisner et al. (1996)
note that “agencies almost universally state that time and resources are too limited to allow for regular,
systematic reviews” (p. 148). While removing the potential tension between working on proposed rules
and evaluating existing rules, some agencies also noted in this survey that 90 days was insufficient for a
thorough review of significant rules. Second, this Memorandum provided more explicit guidance on how
to revise existing regulations, with an emphasis on performance-based and market-based regulatory
mechanisms. The Memorandum maintained the focus on maximizing net social benefits from Executive
Order 12291 and stressed the need for cost-effective implementation. In some cases, however,
statutory authority precluded both consideration of benefits and costs and implementation through
more novel, market-based approaches. President Bush tasked the Council on Competitiveness to
coordinate the 90-day review. The regulatory moratorium was later extended through the end of the
George H.W. Bush Administration, although a variety of rule-makings unrelated to the look-back
proceeded under exceptions for emergency situations, military and foreign affairs, and judicial deadlines
(Furlong 1995; Copeland 2004; Watts 2012).
In their review of the Bush Administration’s regulatory moratorium and retrospective review,
Eisner et al. (1996) note ten policy rationales for retrospective review of a regulation: change in
Administration policy; change in cost/benefit numbers; changes in technology state-of-the-art,
economic situation, or other factors; implementation/enforcement/litigation problems; complaints,
33
suggestions, and petitions; requests for interpretation; exemption requests; overlapping and duplicative
rules; conflicts and inconsistencies; unnecessary or obsolete rules. As evident below in the discussion of
the Obama Administration retrospective reviews, these rationales are employed by many agencies. They
also note some of the challenges to reviews. For example, regulators may have a vested interest in
existing rules and thus may not have strong incentives for revising them. Moreover, some regulators
may be concerned that revising existing rules may represent an admission of error when originally
promulgating the rule (Bull 2014).
In 1993, President Clinton abolished the Council on Competitiveness, rescinded Executive
Orders 12291 and 12498, and issued Executive Order 12866, “Regulatory Planning and Review.” The
regulatory framework under Executive Order 12866 was similar to that under Executive Order 12291,
although the Clinton Administration employed a less stringent standard that benefits should justify
costs, not necessarily exceed them, and recognized the potential role for non-quantified and/or non-
monetized benefits (Hahn et al. 2003). This executive order called on agencies to prepare an annual
regulatory plan. In addition, regulatory agencies had 90 days to submit to the Office of Management and
Budget a plan for periodic review of existing significant regulations. This review would “determine
whether any such regulations should be modified or eliminated so as to make the agency’s regulatory
program more effective in achieving the regulatory objectives, less burdensome, or in greater alignment
with the President’s priorities and the principles set forth in this Executive order” (section 5). The Vice
President also had the authority under this executive order to identify for review existing regulations or
sets of regulations (perhaps issued by multiple agencies) that may affect specific groups, industries, or
sectors of the economy.
In 1995, the National Performance Review led a substantial review of the Federal regulatory
program. As a part of the “reinvention” of government, the National Performance Review worked with
regulatory agencies to identify outdated, obsolete, and inefficient regulations that could be modified or
34
rescinded. Through this effort, agencies proposed to eliminate 16,000 pages of regulations that would
reduce regulatory burdens by about $28 billion per year (National Performance Review n.d., 1993;
Copeland 2004). A key characteristic of the operation of the National Performance Review was the
creation of various reinvention teams, composed of representatives of various agencies, who worked
outside the normal bureaucratic channels to develop ideas and make recommendations for reform.
During the George W. Bush Administration, the Office of Management and Budget issued
“prompt” letters that suggested ideas for agencies to pursue new regulations. In some cases, these
replaced existing rules. Moreover, the Office of Management and Budget solicited nominations from the
public to identify existing rules that merited reform. In response to the 2001 request for nominations,
the Office of Management and Budget received 71 suggestions for review. After a second call for
nominations in 2002, the Office of Management and Budget received more than 300 suggestions
(Copeland 2004). The regulatory agencies initiated efforts to revise approximately 100 rules under this
public-nomination process (Graham et al. 2005; Office of Management and Budget 2004).
Lessons from Past Retrospective Reviews
One of the common themes across these various Presidential-mandated retrospective reviews is
the focus on reducing burdens. This reflected the widely held view of the regulated communities that
they bore unnecessarily high costs. There was much less emphasis on maximizing net social benefits. As
a result, the approach focused on identifying rules to eliminate or streamline.
Each one of these reviews yielded reforms to the regulatory state. Regulations were rescinded
and burdens were reduced during each period of retrospective review. Outdated rules were
modernized. While regulators indicated that heightened attention by political leaders resulted in their
paying more attention to retrospective review (e.g., see survey in Eisner et al. 1996), it is not clear the
35
extent to which the retrospective review mandated by the White House, as opposed to regular agency
operations under existing statutory authority and the Regulatory Flexibility Act, resulted in the rules’
changes. In all of these cases, it is difficult to construct a counterfactual scenario against which to
compare and assess the experience under each review.
The frequency of reviews mandated by the White House may suggest that the regular agency
practice under existing authorities and the Regulatory Flexibility Act are not sufficient to identify rules
that could be revised or eliminated to reduce regulatory burdens (or increase social welfare). In
addition, the frequency of these reviews may reflect political expedience in response to opponents of
government regulations. The Reagan and Bush II reviews occurred at the start of their administrations
and represented a response to the view that the preceding Democratic administrations had been too
aggressive as regulators. The Bush I review, occurring at the start of an election year, and the Clinton
review, occurring in the first year of the Republican-controlled Congress, illustrate the potential political
need to show effort and success in reducing the costs of regulatory policy on the economy (Furlong
1995; Watts 2012).
These efforts employed a wide array of implementation models. In several cases, the
retrospective review called on agencies to develop procedures and undertake their review (subject to
some White House coordination). In the case of the Clinton Administration, the National Performance
Review model served as an example of an outside team (if not fully independent) working with
regulators to identify rule changes. And in the Bush II administration, the process was a fairly narrow,
Office of Management and Budget initiated review of rules. Independence in the retrospective review
has some important merits, especially in terms of credibility and legitimacy (Lutter 1999; Greenstone
2009). Providing an explicit role for the Office of Management and Budget, which takes a broader
perspective and can more easily identify regulatory overlap, regulatory gaps, and regulatory
accumulation, clearly has merit as well. Given the significant heterogeneity in size, mission, and culture
36
of regulatory agencies, there is value in providing discretion to agencies in shaping, at least to some
extent, their retrospective review programs.
37
Proposals for Reform of Retrospective Review
While the vast majority of retrospective review efforts dating to the Carter Administration have
originated and operated within the executive branch, proposals in recent years would call for legislative
action and provide Congress with opportunities to require the elimination of specific, existing
regulations. This section briefly describes and evaluates several of these proposals before turning to an
examination of the Obama Administration’s retrospective review efforts in the followings section.
Regulatory PAYGO
As noted above, President Reagan’s Executive Order 12291 called for the collection of data
necessary to develop a regulatory budget, but this was not meaningfully implemented before President
Clinton rescinded this executive order in 1993. The basic concept is similar to pay-as-you-go budget
procedures on the fiscal side of government activities. Regulatory pay-as-you-go would establish a
“cost” budget for any given agency’s regulatory program, typically based on an estimate of the costs of
its current suite of regulations. In the process of proposing a new regulation, the regulator would have
to identify an existing regulation with same or greater costs imposed on regulated entities for
elimination. Thus, the development of new regulations imposes a discipline of reviewing and striking
existing regulations to ensure that the net cost burden of that agency’s regulatory program does not
change.
Senator Warner (2010) has expressed support for such an approach. Likewise, recent legislative
proposals have included some version of regulatory PAYGO. The “Searching for and Cutting Regulations
that are Unnecessarily Burdensome Act of 2014” (H.R. 4874, 113th Congress; the “SCRUB Act”) includes a
so-called “CUT-GO” provision. In this bill, an appointed commission would identify existing Federal
38
regulations for elimination, with the objective of reducing the aggregate costs of Federal regulation by
at least 15%. This commission would report this recommended list of rules for elimination to Congress,
and each chamber of Congress would have the opportunity to approve of the recommendations through
a joint resolution process. If these recommendations are approved through a joint resolution, then
agencies shall initiate the regulatory process for striking the listed rules. Absent a joint resolution, the
recommended list of rules still imposes a meaningful constraint on regulators. If an agency decides to
promulgate a new rule, it must offset the cost of the new rule by striking rules with equal or greater
costs from the recommended list.
Regulatory PAYGO suffers a daunting technical challenge. As noted above in Harrington (2006)
and Office of Management and Budget (2005), one of the challenges with understanding the economic
impact of the current Federal regulatory program is the dearth of ex post estimates of benefits and
costs. Generating an aggregate estimate of the costs of a given agency’s suite of regulations – especially
given the variations in the timing of costs (some rules impose large capital investments, which are one-
shot investments, while others impose periodic operational costs), potential interactive impacts of
multiple regulations (which could either increase or decrease aggregate costs relative to assessment of
the individual regulations), and even potential interactive impacts of regulations with other agencies – is
very difficult. Moreover, whatever estimate an independent commission would produce would be
subject to quite significant uncertainty, which could be problematic given the precision within which the
estimates would be used in determining whether a new regulation could go forward.
More important, regulatory PAYGO is inconsistent with fundamental principles of regulatory
policy. The government is in the business of regulation to attempt to correct failures in the operation of
markets. A government intervention mitigates the market failure, at least to some extent, if its benefits
exceed its costs, and the intervention should aim to deliver what the markets would produce if they
were not characterized by the market failure. In other words, regulatory interventions should maximize
39
net social benefits. Regulatory PAYGO completely ignores the benefits side of the ledger. Implementing
regulatory PAYGO could make society worse off. Consider an example of two regulations, one existing
and one proposed. Suppose that each regulation has social benefits that exceed social costs. Under the
status quo approach to regulation, the government should implement both the existing and the
proposed regulation. Under regulatory PAYGO, the government would have to eliminate the existing
regulation, with positive net social benefits, if it aims to implement the proposed regulation. This is
contrary to the weak and strong efficiency standards that have guided regulatory review since 1981.13
Regulatory Review Commissions
The idea of an independent commission to evaluate regulations, if guided by a net social
benefits standard instead of the strict cost standard of regulatory PAYGO, has some potential merit. In
addition to the commission envisioned in the SCRUB ACT, the “Regulatory Improvement Act of 2014”
(H.R. 4646, 113th Congress) would establish a commission that would make recommendations for
striking regulations based on their economic costs. These recommendations would be considered in
their entirety by Congress and if approved by each chamber and signed into law by the President, they
would trigger agency regulatory processes for eliminating the listed rules. The process would effectively
mirror the base realignment and closure process for military facilities after the end of the Cold War.
A fresh set of eyes to evaluate regulations, especially by those who do not have a vested interest
in the outcome like regulators may have during their assessment of their own regulatory programs,
could bring substantial value to retrospective review. Nonetheless, attempting to evaluate the entirety
of agencies’ regulatory programs is a task that would clearly require more time than allocated to the
commissions envisioned in the “Searching for and Cutting Regulations that are Unnecessarily
Top Related