Better Regulation for Growth - World Bankdocuments.worldbank.org › curated › en › ...Better...

Better Regulation for Growth Governance Frameworks and Tools

for Effective Regulatory Reform

Project-Level Indicators

In partnership with:

Investment Climate Advisory Services World Bank Group

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

wb371432

Typewritten Text

59924

BRG: Project–Level Indicators

i

About the Investment Climate Advisory Services of the World Bank Group

The Investment Climate Advisory Services of the World Bank Group helps governments implement reforms to improve their business environment, and encourage and retain investment, thus fostering competitive markets, growth and job creation. Funding is provided by the World Bank Group (IFC, MIGA, and the World Bank) and over fifteen donor partners working through the multi-donor FIAS platform.

The findings, interpretations and conclusions included in this note are those of the author and do not necessarily reflect the view of the Executive Directors of the World Bank Group or the governments they represent.

Better Regulation for Growth Program

The Better Regulation for Growth (BRG) Program was launched in 2007 by the Dutch Ministry of Foreign Affairs, the UK Department for International Development (DFID) and FIAS, the investment climate advisory services of the World Bank Group.

The objective of the BRG Program is to review and synthesize experiences with regulatory governance initiatives in developing countries, and to develop and disseminate practical tools and guidance that will help developing countries design and implement effective regulatory reform programs. Reports and other documentation developed under the BRG Program are available at: www.ifc.org/brg


iii

Acknowledgments

This report has been prepared by Claudio Radaelli and Fabrizio de Francesco, University of Exeter.

An early draft of this document was discussed at the Technical Meeting of the Advisory Panel of the Better Regulation for Growth Program held in The Hague on November 13-14, 2008. Comments to this draft were provided by participants to this meeting: Andre Nijsen, Andreja Marusic, Andrea Renda, Claudio Radaelli, Colin Kirkpatrick, Delia Rodrigo, Edward Donelan, Esra Yilmaz, Eva Buitenkamp, Fabrizio de Francesco, Greg Bounds, Jeroen Nijland, Josephine Kanyi, Ksenija Vidulic, Lars Grava, Margo Thomas, Mustafizur Rahman, Nick Godfrey, Richard Sandall, Stephen Rimmer, Scott Jacobs and Ulrich Ernst.

Furthermore, advanced drafts of the report benefited from valuable comments from Delia Rodrigo, Peter Ladegaard, Stephen Rimmer, Lars Grava, Richard Sandall, Adrian Nembhard and Masrur Reaz.


v

Contents

Executive Summary .......................................................................................................... vii

1 Introduction .............................................................................................................1

2 Existing indicators ....................................................................................................3

2.1. An overview of the Monitoring and Evaluation indicators applied for IFC projects .............................................................................................3

2.2. OECD Project-level indicators ......................................................................7

3 Approaching the design: Conceptual insights on indicators ................................11

4 Institutional dimensions: bundles of innovations and implementation issues ......................................................................................................................13

4.1 RIA and other contextual innovations .......................................................13

4.2 Stages of implementation ..........................................................................14

5 Indicators for better regulation for growth projects ............................................17

5.1 Tackling four hard questions .....................................................................17

5.2 Discussion and pathways for analysis ........................................................19

6 Conclusions ............................................................................................................23

Bibliography .......................................................................................................................25

Appendix: Set of Indicators of Regulatory Quality .........................................................29

Annex: Better Regulation for Growth Program ..............................................................65

List of Boxes

Box 1: The 1997 principles of good regulation ...................................................................7Box 2: The 2005 Guiding Principles for Regulatory Quality and Performance ...................8


vi


vii

Executive Summary

In this paper we review the experience with project-level indicators associated with regulatory reform, and propose an approach to design and implementation, drawing on the findings of working paper no.1 “Overview of Regulatory Quality Indicators”.1

The paper examines existing FIAS and OECD indicators and suggests improvements in content, organisation and presentation. For example, a separation of indicators that FIAS can and cannot impact itself is recommended.

One key theme throughout this paper is the link between indicators and knowledge utilization. Another is the concern with the overall architecture of systems of indicators.

The paper then considers the broader issue of how systems of regulatory indicators should be developed. At the outset, we reason that FIAS often operates in a non-canonical scenario, where objectives are vague and difficult to measure, and problem-definition is genuinely difficult or contested. Under these conditions, the rules and routines of the regulatory performance measurement system should emerge from an open discussion with stakeholders. The design of indicators should not be limited to technical properties. The establishment of a system of project-level indicators provides an opportunity to reduce some of the uncertainties of better regulation policies. Indicators dissolve ambiguity, create commitments to processes of knowledge utilization, and are a mechanism for revealing the preferences of the major actors. However, in a non-canonical scenario – this is the big caveat – these steps must involve the stakeholders through the establishment of open processes of consultation and deliberation. Without these learning processes, there will be no ownership of indicators beyond the donor. The role of FIAS technical officers is also particular. Their goal in this context is to balance arguments and facilitate learning, with a focus on gradually transferring ownership of the project to the domestic actors.

Considering that the aim in FIAS projects is not simply to convince the recipient country to adopt regulatory quality tools but to embed it in the regulatory processes, the first challenge is one of integrating better regulation tools with prior innovations in administrative law and regulatory policy. The second challenge is to promote progress from the stage of formal adoption of such tools to ‘deep’ implementation.

On the first challenge and relying on empirical evidence on RIA, we observe that better regulation tools are not a plug-and-play innovations. The nature of the institutional terrain in which better regulation projects are ‘planted’ matters, especially in terms of previous adoptions of appraisal obligations and freedom of information acts. The presence of US-style administrative procedure acts (APAs) and context-rules such as giving reasons and due process is a logical pre-condition for better regulation tools and, in particular, RIA although our results on this dimension come with several caveats given the different nature of APAs across nations. Our discussion is driven by empirical evidence. Up until now, most of the evidence available concerns RIA. In consequence,

1 Policy paper also prepared for the BRG Program.


viii

the tool RIA is taken as a model, together with the necessary central unit for the oversight of the rulemaking process. This model (tool + oversight unit) is also applicable to most of the other tools of better regulation (simplification, standard cost model, and consultation).

On the adoption-implementation challenge, preliminary econometric analysis (Radaelli, De Francesco and Troeger 2008) points to problems, such as bureaucratic efficiency, that are characteristic of the deep implementation stage, whilst political constraints and the ideological commitment of the incumbent are more important in early stages. Having explored the institutional dimension and the adoption-implementation stage, we propose different levels of indicators, drawing on our previous work and adapting it to the reality of FIAS. We argue that the crucial tests for systems of indicators are (a) to distinguish between formal adoption and implementation; (b) to be sensitive to administrative context with its pre-existing requirements and tradition; (c) to balance diversity and the need for convergence; and (d) to link measurement to credible commitments.

The approach to indicators and their usage – we suggest – could usefully be modelled after the notion of layers or types of indicators. Specifically, in the paper and, in more detail, in the Appendix we distinguish between a layer of background indicators and five other layers (or ‘levels’) covering (i) the regulatory reform policy (ii) quality of organizational design (iii) guidance and support (iv) ouput and (v) impact indicators. The rationale for this choice is to promote learning and benchmarking by allowing some degrees of diversity in the initial conditions and administrative law. Measures of better regulation policy, organizational design, and guidance and support can be adopted by all countries that receive aid from the World Bank, whilst the surveys on output and outcome can be used for the most advanced experiences because better regulation programs need several years to be implemented and their impact is not immediate.

It is possible to aggregate indicators by layer and even calculate an overall composite measure covering the five layers. Further work should draw on our knowledge of composite indicators and make the insights of this architectural paper operational.

One question that is often asked is whether the weighting2

2For more details on the aggregation process please refer to De Francesco and Radaelli’s paper on 'Overview of regulatory quality indicators'

should be done by the social scientists or by the policy-makers. The literature suggests that both options are feasible. Our suggestion is that the social scientists define the menu of technically feasible indicators, but the policy-makers choose from the menu by revealing their preferences about weights. Put differently, the selection and aggregation of measures should be usefully left to a process of choice led by policy-makers. Our technical input to this discussion is to suggest a design method based on the layers. Policy-makers should use selection and aggregation to reveal their definition of regulatory quality and sort out their preferences transparently. This does not mean that ‘anything goes’ and that senior bureaucrats and elected politicians should operate within unaccountable processes.


ix

They should be accountable to their citizens about why and how they choose a specific measurement system and what they want to achieve with it. Experts should be the guardians of the quality of the menu, and in our vision play an important role in outcome indicators, which draws on the judgment of professional independent evaluation in order to review, correct and improve (through evaluation standards) the overall policy. International experience with independent evaluation of better regulation policy evidences that such system is feasible also because of the availability of a wide range of methodologies.

This paper is produced as part of the Better Regulation for Growth (BRG) programme. BRG is a joint initiative of the Dutch Ministry of Foreign Affairs, the British Department for International Development (DFID) and FIAS, the multi-donor investment climate advisory service of the World Bank Group. Its objective is to deepen understanding of regulation and regulatory reform, in order that the positive impact of reform on economic growth and poverty reduction can be strengthened.


x


1

1 Introduction

Since the mid-1990s, a political interest in ‘governance’ has emerged among developing countries. It has brought about overall administrative reforms to fight corruption, principles such as rule of law and bureaucratic efficiency as well as economic reforms to attract foreign investment and enhance economic development (Drori 2006). These reform activities have been coordinated by a web of transnational organizations and national agencies (Drori 2006: 91). An important role has been played by international organizations such as the World Bank and the World Trade Organization (WTO). Administrative accountability, regulatory quality and transparency have become key priorities for these organizations, and their diffusion is nowadays often seen as necessary conditions for economic growth and development (Delacroix and Ragin 1981; Bardhan 1997; Goldsmith 1999; World Bank 2002).

Good governance and administrative reforms have also been pursued by developed countries independently from international organizations with the aims of improving the efficiency of public administration and reducing the public deficit. In a sense, ‘the term of governance has come to mean the art of governing, as distinct from the institutions of government’ (Drori 2006: 107).

In this attempt to enhance transparency and performance, governments everywhere are engaged in establishing 'tight, objectified links between objectives, means, output and outcomes' (Noordegraaf and Abma 2003: 853). With the diffusion and implementation of the new public management (NPM) paradigm, there has been an extensive transformation of the language, discourse, symbols and structures of public organizations. This attention towards measures is essentially due to citizens' request to get more value for money and to technological improvement. 'Measurement functions as a source of authority in order to inform, to legitimate and to control managerial decisions... [introducing] new control modes within a performance-oriented climate' (Noordegraaf and Abma 2003: 856). Measurement management has been applied in many public policies: budget, education, health, and social policy. Whilst the paper on “Overview of Regulatory Quality Indicators” mostly analyses the overall performance of governmental regulation and derives measures connected to the economic outcome of regulatory policies, this paper, instead, reviews and proposes specific indicators for better regulation programs. When we focus on a specific project in this paper, we make the assumption of a project consisting of the adoption of an administrative requirement to appraise, for instance, ex-ante regulatory effects via a methodological tool known as regulatory impact assessment (RIA) and the establishment of a central unit in a given country – through support from an international donor – with the main task of designing and exercising oversight on the implementation of regulatory reform. To large extent, however, the same considerations and findings are applicable to other tools and projects for regulatory quality such as simplification, consultation, as well as access and transparency.


2

Theoretically, it is indispensable to focus on specific projects and areas of reform, since the literature has exposed the limitations of comprehensive measures of the performance of governments and the public sector (Van de Walle 2008a; 2008b). But this is also a pragmatic choice given that the Investment Climate Department (IC) is involved in projects and needs information on their results, rather than overall measures of the performance of public administration (for this dimension of measurement, we refer to some of the indicators reviewed in the other paper). Concretely, this paper probes projects-level indicators for developing countries, although we will also draw from indicators we have proposed for appraising regulatory quality in Europe. We are also confident that the methodological proposal for designing indicators can also exploited by other development agencies and regulatory reform officials in developing countries.

The organization of the paper is the following. We start from what is available on the ground: in Section 2 we briefly comment on the indicators actually used by the IC Department at the World Bank Group and the OECD. We find that there is room for improvement. To show how to improve, we approach the design of project-level indicators (in Section 3) by considering the most important technical dimensions of indicators and by relating design to the characteristics of the regulatory reform issues. Here we will introduce the difference between canonical and non-canonical practices, and discuss its implications for the design of indicators. Section 4 introduces two major features, that is, the relationship between project-level indicators and other regulatory innovations and the distinction between the adoption of the project and its implementation. We will argue that project-level indicators should incorporate valuable information on (a) other regulatory reform innovations that, if present, complement and support the project; and (b) the different stages of the policy process. On point (b) we will observe that most regulatory indicators in use track down the early stages of the policy process, such as adoption and the preparation of formal guidance, but tend to miss the implementation stage. This is crucial instead, since when the donor leaves the country, there is a risk of project atrophy if implementation is not monitored. Having shown complexity and why it matters by highlighting four ‘hard questions’ for project indicators designers, we turn in section 5 to our proposal for regulatory indicators of a RIA project, showing different forms of usage and modalities of aggregation. Section 6 concludes.


3

2 Existing indicators

2.1. An overview of the Monitoring and Evaluation indicators applied for IFC projects

The World Bank’s IC Business Enabling Environment (BEE) projects are ‘designed to improve institutional legal and regulatory conditions under which businesses operate, including changes to government policies, laws, regulatory frameworks and administrative practices by public sector entities. The approach covers ‘several stakeholders both within and outside the World Bank group’ (Batra et al. 2005: 1). To exemplify, in the project conducted by the World Bank in Serbia, the BEE assumes an integrated connotation along two dimensions (in turn, both national and sub-national), i.e. quality of institutions and the focus on policy input and results and output achieved. To clarify our choice to focus on World Bank's IC projects is pragmatic. A discussion on the quality of indicators and consequent evaluative process should be based on specific policy objective. The activities on regulatory reform have been interlinked with other policies and outcome measures, such as competition policy, investment promotion and generation, as well as promotion of alternative dispute resolution mechanisms. Within this framework, RIA plays an important role. This is evidenced by the attention that IC has given to this administrative procedure in several projects conducted, for instance in Kenya and Serbia. It is important therefore to understand the approach to performance management.

Monitoring and evaluation activities of regulatory reform projects rest upon 'relevant, meaningful, cost-effective performance indicators to measure the outputs, outcomes, and impact of International Finance Corporation’s technical assistance projects’ (Batra et al. 2005: 1). An essential characteristic of the performance measurement exercise within business enabling environment is the reliance on a wide pool of end-recipients: ‘many if not most of whom will not know what organizations were involved or in many case be able to identify precisely how the project impacted their business.’ (Batra et al. 2005: 2)

The International Finance Corporation (IFC) uses a ‘program logic model’ for assessing and monitoring BEE projects that is composed of three elements: envisioned activities/outputs, intended outcomes and targeted impacts (Batra et al. 2005: 2).3

3 The current M&E framework used in IFC contains for each product a number of specific actions (usually 10 to 15) that, if completed successfully (combined or by themselves), can have a significant role in ensuring that the outcomes of the investment climate advisory services lead to the impacts the organization aims to achieve. These actions are called "reforms" and they are defined as "...an outcome, or a collection of interrelated outcomes that can be clearly linked to a strategic objective of one of the projects which in turn can be linked to a desirable impact...". The achievement of "reforms" is tracked using the outcome indicators.

This is a framework that also features in the OECD literature and our proposal for EU indicators (Radaelli and De Francesco 2007), based on input, output and outcome. Another common feature is the idea to rely on several and different systems of


4

indicators, composed both of objective and subjective measures - in order to capture the different angles of regulatory reform (Radaelli and De Francesco 2007; Batra et al. 2005: 2).

The program logic model is articulated in five specific regulatory products: business entry; business operations; trade logistics; secured lending and business taxation. The ultimate impact measure is the overall productivity and growth of small and medium enterprises. Batra et al. identify also the characteristics of performance measurement indicators that must be relevant, universal, reliable and easy to collect. Moreover, ‘[t]hey should be applicable to the program being monitored, but general enough to offer benchmarking with similar projects in other parts of the world. They should also utilize a methodology that will yield consistent results, and the cost of collecting the information should be economical.’ (Batra et al. 2005: 3)

This paper analyses 25 indicators used by the IFC in relation to its Business Enabling Environment programmes.4

• indicators of general activities

Eight of these measures refer to outputs; thirteen to outcomes; and four to impacts. The first group of measures refers to the activities conducted by the World Bank group. These eight measures can be sub-classified according to the following categories:

o number of entities receiving advisory services

• indicators of training

o number of participants in workshop, training events, seminars, conferences, etc.

o number of participants reporting satisfied or very satisfied with workshops, training, seminars, conferences, etc.

o number of participants in workshops, training events, seminars, conferences, etc.

• indicators related to regulation/regulatory governance

o number of new law/regulation/codes drafted or contributed to the drafting

o number of procedures/policies/practices proposed for improvement or elimination

o number of reports (assessment, surveys, manuals) completed

• indicators of communication

o number of media appearances

The second group of indicators on outcomes can be grouped as follows:

• Indicators developed on the basis of previous indicators

4Excel file detailing these indicators is available from the authors.


5

o Number of entities that implemented recommended changes

o Number of recommended laws/regulation/amendments/codes enacted

o Number of recommended procedures/policies/practices that were improved/eliminated

• Doing Business indicators or related to DB indicators

o Average number of days to comply with business regulation

o Average official cost to comply with business regulation

o Number of businesses completing a new/reformed procedure in a given jurisdiction

o Number of jurisdictions reporting at least one Doing Business reform

o Number of reform resulting from advisory service as measured by Doing Business

• Indicators related to alternative dispute resolution (ADR)

o Number of cases successful through ADR

o Number of days to settle a case through ADR

• Indicators related to investment

o Number of investor inquiries in targeted sectors (Indicators of communication)

o Number of investor inquiries in targeted sectors leading to investment

o Score obtained by Investment Promotion Intermediary on IP performance review

Finally the last four measures of impact:

• Number of formal jobs created

• Value of aggregate private sector savings from recommended changes (US$)

• Value of investment/financing facilitated by advisory services (US$)

• Value of funds released through ADR (US$)

Our comments are as follows:

• As a general rule, we would suggest a different presentation of the indicators, possibly by entering indicators by row and the following information by column: name of the indicator, definition (what is measured), interpretation (how does


6

one interpret changes over time), range, and data-gathering (who and how gathers the data).5

• The three sets of indicators are not logically-conceptually linked. It is difficult to argue that the indicators used so far follows the causal link that better regulation programs increase of the quality of regulation and consequentially increase of productivity and economic growth. Beyond the consideration on the feasibility and sensibility of the impact indicators, what is missing is a clear reference to the improvement of regulatory process and a better measurement of regulatory environment that is captured only through the DB indicators and few other indicators such ADR and business investments. Further, measures of the industry sectors' productivity are lacking.

• Some improvement on crude numbers is obviously possible by using ratios, for example consider number of persons/hour trained.

• Another critical aspect of the indicators is that some of them are not under control of IC Department-World Bank. It would be better to separate out those indicators on which the IC Department can intervene from the others on which other actors operate. Indicators of general activities and indicators of training are examples of the first category, whilst the outcome and impact indicators not only measure the quality of the intervention but also the capacity of the beneficiary countries' public administration.

• The measures seem to be insensitive to the decision-making process of the country (although this background information may be available through other indicators available at IC Department, it should be integrated). Neither is there room in these indicators for information on the administrative context, how rule-making is disciplined and the type of legal culture.

• We will explain throughout the paper that adoption and implementation are two different processes, and need different measures. This is not captured by the current indicators.

• There is no discussion of the role of the government in collecting data for indicators and the capacity to provide data and information.

• Value-for-money does not seem to feature in the design and interpretation of indicators, neither does the scheme provide information or caveats on the possible bias of individual indicators and how the bias can be corrected.

• Overall, the indicators need improvement in order to provide a valuable tool for the evaluation of the better regulation projects.

5This is the approach we followed in Radaelli and De Francesco (2007) further to a suggestion made by Scott Jacobs. We wish to stress the importance of this presentation of the indicators. Once we adopted it, this presentation made us think twice about what we exactly meant and increased the potential for valuable communication with our client (at that time the European Commission) and the stakeholders.


7

General programme indicators used at World Bank/IFC do not mention a number of important contextual features, such as the administrative system, the regulatory process of a given country (and related administrative requirements), and seem to ignore the distinction between adoption and implementation of better regulations. It is neither clear how one can tell when and how the quality of the regulation has improved. Doing Business, business surveys on the satisfaction with regulatory environment, or even measures derived from the implementation of the Standard Cost Model may be more informative than this set of indicators.

2.2. OECD Project-level indicators

Indicators have been widely used in the OECD context. With the goal of promoting the adoption and the effective implementation of RIA, the OECD published in 1997 a report on best practices, drawing the recommendations in box 1, to be considered jointly with the 2005 principles (Box 2). Such recommendations have been the basis for designing checklists, composed of Y/N format questions. In turn, although principles are not indicators, the YES-NO answers can be used for rudimentary indicators and checklists. The principles are often used as benchmark guideposts for the peer review processes undertaken by the OECD member states and as a basis of the OECD regulatory reviews of individual countries.

Box 1: The 1997 principles of good regulation

1. Adopt at the political level broad programs of regulatory reform that establish clear objectives and frameworks for implementation.

2. Review regulations systematically to ensure that they continue to meet their intended objectives efficiently and effectively.

3. Ensure that regulations and regulatory processes are transparent, non-discriminatory and efficiently applied.

4. Review and strengthen where necessary the scope, effectiveness and enforcement of competition policy.

5. Reform economic regulations in all sectors to stimulate competition, and eliminate them except where clear evidence demonstrates that they are the best way to serve broad public interests.

6. Eliminate unnecessary regulatory barriers to trade and investment by enhancing implementation of international agreements and strengthening international principles.

7. Identify important links with other policy objectives and develop policies to achieve those objectives in ways that support reform.

Source: OECD Report on Regulatory Reform, 1997.


8

Box 2: The 2005 Guiding Principles for Regulatory Quality and Performance

1. Adopt at the political level broad programs of regulatory reform that establish clear objectives and frameworks for implementation.

2. Assess impacts and review regulations systematically to ensure that they meet their intended objectives efficiently and effectively in a changing and complex economic and social environment.

3. Ensure that regulations, regulatory institutions charged with implementation, and regulatory processes are transparent and non discriminatory.

4. Review and strengthen where necessary the scope, effectiveness and enforcement of competition policy.

5. Design economic regulations in all sectors to stimulate competition and efficiency, and eliminate them except where clear evidence demonstrates that they are the best way to serve broad public interests.

6. Eliminate unnecessary regulatory barriers to trade and investment through continued liberalization and enhance the consideration and better integration of market openness throughout the regulatory process, thus strengthening economic efficiency and competitiveness.

7. Identify important linkages with other policy objectives and develop policies to achieve those objectives in ways that support reform.

Source: OECD Taking Stock of Regulatory Reform, 2005.

Interestingly, in the context of the SIGMA project supporting the Central and Eastern European countries into their accession to the European Union, the OECD has acknowledged that

‘[t]here is no universal or unique recipe for achieving Better Regulation. The basic ingredients of a Better Regulation policy are discernable from principles and policies at work in the EU and in many OECD Member States. These ingredients are:

Sustained political support for Better Regulation, it will not come about by accident,

The traditional monopoly of lawyers in the rule-making process is no longer justified, improving regulation can only be achieved by a multidisciplinary approach across government, it requires to go beyond a mere legalistic point of view in order to take into account the economic, environmental and social impact of legislation,

Better Regulation is not achieved by a one-shot policy but is the result of a continuous process of improvement aimed at both the flow of new regulations and the stock of existing ones,

Better Regulation is best achieved through a series of specific targets that bring tangible results in a discernable time period


9

Better Regulation is best achieved by a ‘bottom up, top down’ policy with a central body playing an advocacy and support role to line ministries, who much also commit to a process of continuous improvement of regulations and the regulatory process,

The effective application of common tools: consultation, impact assessment, accessibility and simplification.’ (OECD-SIGMA 2006: 44-5)

Overall, recognizing the complexity of achieving effective implementation because of the lack of a single, universal model for achieving regulatory quality, there has been a refinement of the OECD recommendations that require a steady process of commitment and learning, made of oversight and ownership of regulatory reform as well as specific targets for monitoring the progress. Policy outcomes are achieved in the long term, with relevant investment. Dimensions of governance must be taken into account in the establishment of policy and tools. Determinants of successful institutional innovations are broader, complex and context-bound,6 taking into account 'a web of politically consequential institutions and institutional relationships inside the state.' (Williams 2002: 397).7

Some of these important issues are dealt in the remainder of this paper. We step back and look at how to build indicators, following broadly shared models or templates. We then tackle the institutional dimension, specifically by considering two types of scenarios in the final part of section 3.

In other words, what is still lacking in the OECD's discussion of adoption and implementation of market-oriented reforms (such as regulatory reform and RIA) is the consideration and analysis of institutions and bureaucratic behaviour. Notwithstanding, these critical remarks the approach of the OECD is interesting for project that want to establish principles of regulatory quality and on which build up comparative and review mechanisms to foster learning among group of donor countries.

6On the implications for RIA see Radaelli (2005), who draws on historical institutionalism. 7By institutions relationships is meant 'formal and informal patterns of governance, decision rules, standard operating procedures and ministerial mandates'; by institutional relationships is meant 'the arrangements based on institutions that constrain state actors, and structure their relations with one another' (Williams 2002: 397).


10


11

3 Approaching the design: Conceptual insights on indicators

Within a management and measurement cycle, indicators refer to the specific steps of monitoring (performance measurement) and evaluating (comparison between actual and intended performance). The technical discussion on how to select appropriate project indicators is now well developed in the literature, and revolves around the quality of performance measurement systems and the characteristics of indicators (Radaelli and De Francesco 2007). Objective indicators, characterized by a straightforward link between cause and effect, are however an ideal type. The reality is more complex. Information is uncertain or not completely available and ultimately there is 'imprecision in estimates of future consequences conditional on present actions' (March 1994: 174). In a bounded rationality framework, information management is 'never wholly clear, consistent and orderly'. Accordingly, 'managerial behavior is not just about collecting and using information, but about interpreting information' (Noordegraaf and Abma 2003: 859). In this interpretation of the reality, indicators require shared definitions.

Given these different theoretical assumptions, the key variables of an evaluation process are a) the extent to which issues are known; b) the extent to which issues are contested and standards need to be agreed (Bowker and Star 1999, cited by Noordegraaf and Abma 2003: 863-4). Specifically, canonical practices8

The implications for the activity of the IC Department in many different countries are straightforward. Most of the target countries are bound to fall into categories that are closer to the non-canonical scenario. Consequently, the establishment of a system of project-level indicators provides an opportunity to reduce some of the uncertainties of better regulation policies. Indicators dissolve ambiguity, create commitments to processes of knowledge utilization, and are a mechanism for revealing the preferences of the major actors, often made opaque by better regulation rhetoric. However, in a non-canonical scenario – this is the big caveat – these steps must involve the stakeholders through the establishment of open processes of consultation and deliberation. Without these learning processes, there will be no ownership of indicators beyond the donor. The role of IC technical officers is to balance arguments and facilitate learning, with a focus on gradually transferring ownership of the project to the domestic actors and officials.

are characterized by low uncertainty since protocols, training programs and task-descriptions provide clear directions for actions. Non-canonical practices instead are characterized by complexity, inconsistencies and dilemmas, leading to high degrees of ambiguity. In this context, a performance measurement system should derive from an open discussion with stakeholders. It should be designed in a way to consider the multiple stakeholders' perspective and its process should set standards (Noordegraaf and Abma 2003: 867).

8Canonical practices are accepted as being accurate and authoritative.


12


13

4 Institutional dimensions: bundles of innovations and implementation issues

In the previous section we have argued that design should take into account some fundamental properties at levels of indicators. We argue that the design of indicators is not limited to its technical properties. It also involves the delicate communication and networking efforts of the donor within domestic institutions and with the main stakeholders. The first challenge is one of integrating a reform programme with prior innovations in administrative law and regulatory policy. The second challenge is to promote progress from the stage of formal adoption of reform to ‘deep’ implementation. Looking at the case of RIA, in this section we look at these two institutional challenges and discuss how they reverberate on the design of indicators. In section 5 we will then wrap these considerations together and make a proposal for project-level indicators.

4.1 RIA and other contextual innovations9

A vast literature on cross-national adoption of policy innovations has identified prerequisites or necessary conditions for diffusion (Collier and Messick 1975; Bennett 1997). No administrative mechanism (such as the requirement to appraise the economic impact of regulation) can function properly without a broader administrative regime. In the USA, RIA is a component of the broader 1946 administrative procedure act (APA). A crucial aspect of APA is the requirement to provide reasons for proposed regulations and the notice-and-comment obligations. External access to government documents (via the Freedom of Information Act) seems to be another essential condition for the adoption of RIA. Finally, a large number of OECD countries has experienced with Environmental Impact Assessment (EIA) long before RIA was introduced. Thus, the hypothesis can be made that EIA creates some administrative capacity and attitudes towards economic analysis and cost-benefit analysis (CBA) that can be exploited when RIA is introduced. To sum up, RIA as ‘innovation’ can be more or less successful depending on the presence or absence of factors such as the regulation of the administrative process (an APA), the existence of statutory rights to information and access to regulation (FOIA-type documents) and EIA-type innovations.

For the purposes of IC’s activity, it would be most useful to check these propositions on a large set of developing countries. However, it was impossible to find data on a suitable sample of developing countries. We therefore base our discussion on the findings on OECD countries. Notwithstanding difficulties in finding a common definition and purpose of administrative procedure act across countries (Pigott 2002: 380-1), preliminary empirical findings (De Francesco 2008) show that the previous adoption of innovations of good administrative governance is generally a necessary condition, especially among the later adopters. In other words, there is prima facie evidence that

9Section 4.1 is entirely based on De Francesco (2008).


14

RIA is not an isolated innovation but it is inserted in a broader cycle of administrative modernisation composed of multiple, contingent and complimentary innovations. Further, the nature of the institutional terrain in which RIA and other better regulation tools projects are ‘planted’ matters, especially in terms of previous adoptions of environmental impact assessment obligations and freedom of information acts. The presence of US-style APAs (e.g., broader or softer versions) and framework rules such as giving reasons and due process is a logical pre-condition for RIA, although our data on this dimension come with several caveats given the different nature of APAs across nations. Having established that these institutional dimensions matter, we have not said much on the necessary and sufficient conditions for successful RIA implementation. We will suggest a template for analysis based on qualitative comparative analysis later in the paper. Now, however, we need to gain insights on the politics of RIA implementation. These theoretically and empirically founded insights are reflected in the first level of indicators. This refers to the political-administrative context that is not composed of direct measures of the quality of better regulation program. However, these indicators are essential for understanding the broader context in which the project is inserted.

4.2 Stages of implementation

Why do some countries go for a more sophisticated approach to RIA and others prefer a leaner option, based on formal adoption without much implementation? Most datasets measure the adoption of RIA, but there are very few measures of implementation. A recent paper by Jacob et al. (2008) provides some evidence that even in the European Union (EU) the degree of implementation varies to a large extent – so much so that they speak of an adoption-implementation gap.

To get to grips (conceptually, if not empirically)10

Concerning adoption of a better regulation project like the establishment of a RIA system (but this argument can be extended also to other regulatory quality tools), the donor/international organizations and domestic politicians (and their top advisors) are often the main characters, whilst bureaucrats play a minor role. One can argue that politicians want ‘better regulation’ for two reasons. Firstly, better regulation increases the legitimacy of the regulatory system, and this may have a positive impact on the popularity of the incumbent. Secondly, in open economies better regulation increases the competitiveness of a country. A good regulatory environment increases foreign

with implementation, we must derive some hypotheses about the political costs and benefits along the continuum that goes from formal adoption to successful or ‘deep’ implementation. Here it is useful to focus on the political economy of this process from adoption to implementation – following Moynihan (2005) – looking at the preferences of politicians in office and the bureaucracy.

10See Radaelli, De Francesco and Troeger (2008) for a preliminary empirical test.


15

direct investment and deters domestic firms from moving some high value functions abroad.

To adopt (formally) RIA creates benefits to elected politicians. They can show to international organizations that they are following the bandwagon of modernization. Domestically, they send to the business community a signal that they are doing something to improve on the regulatory environment. The economic and political benefit of saying YES to the OECD 1995 ministerial declaration on regulatory quality – to illustrate with an example – is high, the cost is low.

The next step is to produce guidelines on RIA. This comes at moderate administrative cost (the senior civil service has to coordinate views and ‘model’ impact assessment as a process with specific steps, such as problem definition, consultation, economic analysis, choice of options, and monitoring). Politically, the core executive sends a signal to departments that their regulatory activity may be watched closely. In coalition and/or minority governments, this has political costs – some members of the coalition may object to this, especially if they have the regulating departments in their portfolios.

Next comes the stage of putting money on the enterprise. Guidelines do not work without proper investment. The core executive, therefore, has to invest in resources, such as training, hiring specialists in the economic analysis of regulation, and staffing departments with economists. This has a clear economic cost. In departments calibrated around lawyers and generalists, the addition of economists can also create cultural friction and therefore some political costs. The benefits, of course, are all in terms of having more chances of controlling from the centre the regulatory activity. The most evident sign of commitment is the establishment of a central unit with its own staff and budget.

Then, implementation reaches the stage of actually carrying out proper impact assessments. This has high economic costs – some major RIAs are quite sophisticated, they take time and require different types of models and analysis. There is also a political cost – the core executive has to exercise pressure on departments that are not so keen on RIAs. The political benefit is that only if RIAs are systematically produced, can the core executive control the regulators.

Finally, there is the step of publishing the RIAs widely, including the analytic documents (such as quantitative economic analysis) on which the final RIA was based. This increases transparency in the regulatory process. This step may not cost much in terms of economic resources – depending if publication is limited to the internet or includes the official Journal or Gazette. But clearly it has a political cost, since all affected interests (not just the ones that are within the constituency for support of the incumbent) can use the RIA as information-device and organize their pressure on the regulators. So the cost is about the overall uncertainty in terms of who will get what out of the RIA.

To recap, formal adoption does not involve major political and physical costs but mainly serves as a signal to international organizations and capital markets that the country is trying to provide a favourable regulatory environment. The second stage of


16

implementation has real budgetary implications and therefore should not only be affected by political but also economic considerations. The production and publication of RIA results is both a political and a bureaucratic issue. The timely production of RIA is contingent on the efficiency of the bureaucracy as is the publication and circulation-publication of the results of individual impact assessments. In addition, interest groups should be highly concerned with RIA results in order to monitor agency's decision making processes and use it as an information device on how regulatory costs and benefit are being allocated through the choice of one option or another.

Our preliminary econometric analysis (Radaelli, De Francesco and Troger 2008) points to problems such as bureaucratic efficiency within the deep implementation stage, whilst political constraints and the ideological commitment of the incumbent are more important in early stages. The ideological ‘color’ of the government does not matter for successful implementation – showing that the determination to go all the way down in implementing RIA is not an ideological choice, although the initial adoption of RIA may be seen as right-of-center de-regulatory choice. In the third ‘deep’ stage of implementation the aspects of utmost importance are the efficiency of the bureaucracy and the strength of interest groups. Efficiency of the administration directly influences the probability that RIAs are carried out in a timely and efficient manner and that valid and reliable results are published. The higher the conflict potential between the policy makers and the specific interest group, the higher the pressure to produce impact assessments and publish their results because these results can then be used as detailed information of who is bearing the costs of regulatory choice and where the benefits are going to be allocated.

Being aware of the different and specific political-administrative interactions, efforts, and costs behind each of the adoption-implementation stages provides useful insights on the reality of the development of the better regulation policy. Indicators should try to capture the peculiarities of each stage. This is reflected in our indicators - grouped according to the levels of implementation activities.


17

5 Indicators for better regulation for growth projects

5.1 Tackling four hard questions

In the previous sections we argue that designers of regulatory quality measures face the issue of situating indicators within a problem-definition of the politics of regulatory reform. Level of conflict, uncertainty, shared diagnosis of the regulatory failures or lack thereof, are all variables that affect how indicators are ‘processed’ by actors (see also Radaelli and Dente 1996). There is also the issue that one cannot ‘plug-and-play’ impact assessment or other regulatory innovations onto an administrative rule-making system without considering other institutional innovations, such as administrative procedure acts and the presence or absence of EIA and FOIA-type requirements. Finally, the politics of adoption is very different from the politics of implementation. Having explored these issues, we can now present four hard questions that face the designers of regulatory quality indicators:

1. How does one design indicators that are sensitive to context which includes the administrative pre-conditions for the successful implementation of RIA programs?

2. Secondly, how can one design indicators that distinguish between formal adoption, written content of the guidelines, and administrative procedure standards, on the one hand, and implementation (covering both output and impact), on the other? Imagine you are interested in assessing the quality of the French movie industry: would you rate handbooks on film-making or go to the movie and see a number of recent French films?

3. Thirdly, there is the challenge of respecting diversity and promoting common progress. Across nations, better regulation policies pursue legitimately different – often equally desirable - goals. In a group of countries, this raises the issue of the appropriateness of aggregation and league tables. We argue that policy-makers (not social scientists) should select specific indicators and take decisions about aggregation in the context of a process of coordination between the donor and aided countries (details in Radaelli and De Francesco 2007).

4. This takes us to the fourth challenge: indicators can become tick-the-box routines or have an impact on the interaction among better regulation actors. The decisive factor is the linkage (or lack thereof) between indicators and credible commitments to processes of knowledge utilization. Indicators dissolve ambiguity, reduce the elusiveness of ‘better regulation ideas’ and enable domestic policy makers to make credible commitments only if there is a process in which they are used. Otherwise they are at best an exercise in comparative statistics. For the designer, the challenge is therefore double: to design indicators and to contribute to the emergence of a process to which the key domestic reform actors are committed. At the international level, indicators work if there is a process of facilitated coordination in which policy makers


18

discuss, report, deliberate and ultimately learn about regulatory reform. This process therefore becomes a way to learn about each other’s goals and results, to report on the reasons behind the changes of indicators, and to use socialization to common norms11

Given the complexity of the issue and the lack of a single normative standard on how to conduct an evaluation of better regulation programs, it is useful to distinguish indicators according to the following dimensions:

to create convergence over time.

• Stages of policy process (indicators for input, output and outcome).

• The source of perception (objective vs. subjective indicators).

• The type of evaluator, distinguishing between ‘internal evaluation’ and a system for ‘external evaluation’.

• The better regulation project or tool (RIA, simplification, consultation and simplification).

• The extent of institutionalization of the better regulation project, distinguishing between countries with long experience and the ones with relatively short experience.

Our suggestion is to start from a range of simple indicators, without attempting to perform any kind of ex-ante aggregation.12

Turning to the stages of policy process, three layers of indicators are dedicated to the measurement of project input. This is because the careful design of a better regulation program, its organizational consequences, and the activities of guidance and support are essential for avoiding the implementation gap and ensuring a smooth and effective implementation. The output level indicators require the examination and ex-post measurement of the quality of better regulation tools. To carry on with the metaphor of the cinema, we go to the movies and see some French films (the better regulation tools). But this is not enough. If we are after the impact of films, and not only how technically good they are, we need to measure their real-world impact too. Impact level indicators include survey-measures of regulators, stakeholders, and overall measures of real-world outcome. This level is essentially a bridge between indicators and the more complex task of evaluating better regulation policy (including the evaluation of indicators). Policy evaluation goes beyond indicators since it tackles the question

More fundamentally, we argue, aggregation is possible only if there is convergence on the objectives, principles and purposes of better regulation programs and tools. Normally, this high degree of convergence does not exist ex-ante, but can be reached by using indicators within processes of facilitated coordination.

11Good governance, good regulation, and other norms cannot be defined scientifically but make sense

only in a social context of shared beliefs (Van de Walle 2008a; 2008b) 12For a brief overview of the aggregation process, refer to the paper 'Overview of Regulatory Quality

Indicators'. Other scholars have attempted to aggregate measures for RIA (Visaggio 2007; and for a discussion on composite measures derived from surveys of governments see Jacobzone 2007).


19

whether a policy is effective in dealing with collective problems. Whilst previous levels can be managed by central units or units with an arm’s-length relationship with the government, such as the Better Regulation Group in the Netherlands13

There are several evaluation studies and policy appraisals under way or recently completed on the quality of impact assessment in Europe and the USA. They use a combination of scorecards and more sophisticated tools such as case studies (on the EU, see Renda 2006; Jacob et al. 2008; The Evaluation Partnership 2007). Indicators to be used in external evaluation make use of the subjective judgment of the evaluator or complex calculations where there is a considerable degree of subjectivity in how one compiles the data. External evaluation should also contain an explicit appraisal of the quality of the indicators arising out of open coordination processes and make non-binding suggestions for improvement. Put differently, level of impact adds a reflexive dimension to the indicators. Thus, the technical appraisal of indicators should be conducted within external policy evaluation, whilst the political judgment should remain in the hands of policy-makers.

, policy evaluation is performed by external parties, typically professionally trained evaluators and academics.

In the paper “Overview of Regulatory Quality Indicators” we have accounted for the experience of two independent reviews of better regulation policy conducted by private consultancy companies (in Canada and for the European Commission) as well as several studies conducted by national audit offices (Canada, the UK, and the US). Thus, the idea of an external evaluation is not completely novel - several different methodologies are now available.

5.2 Discussion and pathways for analysis

Turning to the third hard question, the World Bank typically operates with projects such as establishing a regulatory reform unit in a country, with the mission to adopt and implement specific tools, such as simplification programs and RIA systems. Other projects involve several countries. This is the case of large comparative projects, such as Doing Business. When several countries are involved, the approach to indicators and their usage – we suggest – could usefully be modelled around major layers of indicators. We distinguish between a layer of background indicators and five other layers (or ‘levels’) covering

(i) the regulatory reform policy

(ii) quality of organizational design

(iii) guidance and support

(iv) output and

(v) impact indicators.

13See www.ez.nl/english/Subjects/Reduction_of_rules/Reducing_the_regulatory_burden/Contact


20

Appendix one contains examples and details on this approach. One rationale for this choice is to promote convergence by allowing some degrees of diversity in the initial conditions and administrative law. Accordingly, the background layer covers the political-administrative context. The indicators included in the layers of better regulation (or, broadly speaking, regulatory reform) policy, quality of organizational design, and guidance and support can be adopted by all countries that receive aid from the World Bank, whilst the other remaining level output and impact level indicators can be used for the most advanced experiences (among the OECD member states for instance).

Turning to the methodology of data-gathering, the layers of better regulation (or regulatory reform) policy and organizational design are flexible enough to allow a broader agreement on the principles. Specifically, the design and collection of indicators for these two layers should start with the selection of a common set of measures among the donors and beneficiary countries, so that measures and their utilization proceed along the same path. On the other hand, only some countries would be able to adopt measures of guidance and support immediately, but this could be set as a three-year target. Finally, output and impact indicators – the last two layers detailed in appendix one - should target evaluations in the individual countries and assist IC-led discussion fora for the appraisal of indicators. These two layers are therefore suitable for major reviews – this is why data for this system should be collected every three years or so. We stress the importance of an external and independent review and evaluation. This would allow a professional community to grow around the theme of regulatory reform. In turn, such professional community reinforces the legitimacy of regulatory reform.

A simplified way to consider some of the issues raised by the hard questions without adopting a large number of indicators is to aggregate the most important measures by layer. If this is approach is pursued ‘from the side of the donor’, the World Bank and the donor's government will select the ‘most important measures’. By contrast, should the initiative involve a pool of countries, deliberation involving at least the minimum winning coalition for reform14

Some indicators are likely to be relatively stable over the years – administrative law does not change frequently. Consequently, political attention can be focused on indicators that change more frequently, whilst the relatively stable indicators would still be useful for large-n international comparisons and would provide a report on country's lawmaking for those who have not intimate knowledge of the country in question.

would be needed to select the most representative measures.

Several of the proposed indicators rely on datasets mapped in paper 1. In particular, because of the difficulty in clarifying the causal mechanism that link better regulation

14We stress this because it is not indispensable and advisable to look for the consensus of all the major domestic actors in processes of regulatory reform. The necessary conditions are to persuade a minimum winning coalition of actors (for example, the core government departments, the prime minister, a majority of MPs, key federations of business, and responsible civil society organizations) and to generate a legitimizing discourse that effectively communicates the vision behind better regulation.


21

policy to economic growth15

Beyond these global and comparative indicators on regulatory governance and business environment, more specific indicators to compare better regulation systems are extremely useful. In this context, we believe that an OECD type of indicators of regulatory management system could be extremely useful. Most of the proposed indicators in layers 2, 3, and 4 require necessarily the participation of central governmental units in the process of data gathering (especially after the termination of IC programs).

, we rely on the World Bank’s indicators of governance and institutional quality as well as Doing Business that contain the most advanced measures of outcome. Moreover, datasets on independent regulators in developing countries (NERA 2005; Wallsten et al. 2004) may discern which countries (with a more developed approach to regulatory quality) are ready to progress towards the utilization of the indicators contained in layers 4, 5 and 6. Revised and improved versions of the BEE indicators described above could also easily be used. This is because the BEE indicators are more project-oriented that the general DB indicators. Indicators related to foreign direct investment could be aggregated as a composite measure (indicator No. 7 of impact level indicators). Indicators on training could be aggregated in a single measure for the layer “guidance and support” (see indicator No. 4).

15Cfr. Paper “Overview of Regulatory Quality Indicators for a discussion on this.


22


23

6 Conclusions

In this paper we have reviewed the experience of the IFC's IC with project-level indicators, and have made proposals for an approach to design and utilization of systems of indicators. Drawing on previous research by De Francesco (2008) and Radaelli, De Francesco and Troeger (2008), we have also identified the importance of prerequisites and complimentary innovations as well as different stages in the adoption-implementation gap. These empirical findings have fed into our suggestions for systems of indicators – originally developed with the EU context in mind. The proposed levels of indicators can assist policy makers along the stages from adoption to implementation.

In this concluding section, we wish to draw attention to some important points and the forward-looking implications of this paper. The first lesson to draw arises out of the analysis of the connectedness of innovations. Since individual projects are strongly connected to pre-conditions and complementary innovations, it would be wrong to plan the introduction of, say, a RIA system without taking a broader view inclusive of administrative procedure acts, judicial review, freedom of information, environmental policy appraisal and so on. This argument has been captured in the first layer of political-administrative context indicators with a set of preconditions such as the existence of standards for rulemaking process and handbook on policy evaluation. It is a kind of forest and trees argument. It is also an argument about the limitations of thinking of plug and play RIA systems, knowing that they plug and play much better if the pre-conditions in administrative law are there. Finally, it is also an argument about wise resource allocation. Instead of investing, say, 100% of the donor’s resources for country A in RIA, it is more useful to invest, for example, 50% in RIA and 50% in administrative innovations that complement RIA.

The second, related conclusion is about using indicators in order to re-define the scope of projects. We have the impression that some projects are conceived as individual initiatives that do not take into consideration administrative law and the political context. Our indicators relate the tree of the project to a much bigger forest, and assist policy makers in identifying the relevant connections. A better regulation project will succeed or fail depending on how it connects to the discipline of rule-making provided by administrative law, the presence or absence of formal requirement to consult and protect freedom of information, and how better regulation tools alters the balance between core governmental structures and regulating departments and agencies. The lesson from the indicators suggested in this paper is: when necessary re-define the scope of the individual projects – for example by expanding RIA projects to include administrative law and, certainly more problematic but equally important, a re-calibration of core structures in government. This is also a lesson about the linkages between design and utilization of indicators, since the two aspects feed on each other. Inconsistency between the technical design of indicators and their usage led to under-utilization or mis-utilization of indicators.


24

We draw a third conclusion from our analysis of the adoption-implementation gap. The risk is to use indicators that monitor adoption and the very early steps of implementation, but do not track down the further levels of implementation. Crucial here is what happens when the donor goes home: has the government in the recipient country developed ownership of the project? Is there enough administrative capacity to run the system? Who does strategic and operational management once the donor has gone? This has led us to consider different indicators for different stages of the project’s life-cycle, including evaluation – and to highlight institutionalization as one of the main project dimensions to keep under control.

This leads us to a fourth lesson. Granted that individual measures are limited, it is important to avoid the mistake of churning out long lists of indicators without following an internal logic. Systems of indicators are useful if they contain their own internal hierarchies or steps, if there is a sense of logical progress between one element to the system to another, and if it is clear who should use which parts of the systems and when. We have proposed six levels of indicators that can also be seen as a single system with six articulations inside.

Usage of systems varies – depending if the core goal is common progress, convergence led by learning or to let countries experiment as they see fit. As we argue on the paper 'Overview of Regulatory Quality Indicators', there is a continuum of possible forms of utilization, from monitoring to experimental learning. In a sense, this is a trade-off between wholesale convergence of regulatory policies and in drawing lessons that travel from, say, a better regulation project in country A to RIA projects in countries B, C and D. We have suggested how qualitative comparative analysis can support some of these utilizations, but intensive case studies based on project-level indicators can also provide insights on how/why most similar countries in the same region produce different regulatory policy outcomes.


25

Bibliography

Batra, G., M. Bardini and B.R. Weber (2005) ‘Measuring Performance in Business Enabling Environment Projects’, Paper presented at International conference on reforming the business environment, Cairo, Egypt 29 November – 1 December 2005.

Bardhan, P. (1997) ‘Corruption and Development: A Review of Issues’ Journal of Economic Literature, 35:3.

Bennett, C.J. (1997) ‘Understanding the ripple effects: The cross-national adoption of policy instruments for bureaucratic accountability’, Governance, 10:3, 213-33.

Bowker, G.C. and S.L. Star (1999) Sorting things out, Cambridge, MA: MIT Press.

Collier, D., and R.E. Messick (1975) ‘Prerequisites Versus Diffusion: Testing Alternative Explanations of Social Security Adoption’ The American Political Science Review, 69:4, 1299-315.

De Francesco, F. (2008) ‘Prerequisites of adoption and patterns of diffusion: the Case of regulatory impact analysis in European Union and OECD member states’, Paper presented at the 58th

Delacroix, J. and C. Ragin (1981) ‘Structural Blockage: A Cross-National Study of Economic Dependency, State Efficiency, and Underdevelopment’, American Journal of Sociology 86, 1311-47.

Political Studies Association Annual Conference, , 1-3 April, Swansea University.

Drori, G.S. (2006) ‘Governed by Governance.’ In Globalization and Organization. World Society and Organizational Change, edited by G. S. Drori, J.W. Meyer and H. Hwang, 91-118. Oxford: Oxford University Press.

Goldsmith, Arthur A. (1999) ‘Africa's Overgrown State Reconsidered: Bureaucracy and Economic Growth’, World Politics 51:4, 520-46.

Jacob, K.; Hertin, J.; Hjerp, P.; Radaelli, C.; Meuwese, A.; Wolf, O.; Pacchi, C. and Rennings, K. (2008) Improving the Practice of Impact Assessment, EVIA (Evaluating Integrated Impact Assessments) project report. Project No. 028889, European Commission 6th Framework Programme.

Jacobzone, S., C. Choi and C. Miguet (2007) ‘Regulatory Management Systems across OECD Countries: Indicators of Recent Achievements and Challenges’, OECD Working Papers on Public Governance, 2007/7, OECD Publishing.

Kagan, R. A. (2001) Adversarial Legalism: The American Way of Law. Cambridge, Mass: Harvard University Press.

Mayoux, L. (2002) What Do We Want to Know? Selecting Indicators, Manchester: Enterprise Development Impact Assessment Information Service, Institute for Development Policy and Management (IDPM), Manchester University, and


26

Women in Sustainable Enterprise Development Ltd (WISE Development), www.enterprise-impact.org.uk/informationresources/toolbox/selectingindicators.shtml.

March, J.G. (1994) A Primer on Decision-Making, New York: The Free Press.

Moynihan, D.P. (2005) ‘Why and How Do State Governments Adopt and Implement "Managing for Results" Reforms?’, Journal of Public Administration Research and Theory, 15:2, 219-43.

NERA (2005) Regulatory Transparency: International Assessment and Emerging Lessons A Final Report for the World Bank, London: NERA Economic Consulting.

Noordegraaf, M. and T. Abma (2003) ‘Management by Measurement? Public Management Practices Amidst Ambiguity.’ Public Administration 81:4, 853-71.

OECD (1997) Report on Regulatory Reform – Thematic Studies, Paris: OECD.

OECD (2002) Regulatory Policies in OECD Countries: From Interventionism to Regulatory Governance, Paris: OECD.

OECD (2005) Taking Stock of Regulatory Reform: A Multidisciplinary Synthesis. Paris: OECD.

OECD-SIGMA (2006) ‘Assessment of Regulatory Management Capacities of the Slovak Republic’, Paper prepared by SIGMA, Paris: OECD.

Pigott, C. A. (2002) China in the World Economy: The Domestic Policy Challenges, Paris: OECD.

Radaelli, C.M. (2005) ‘Diffusion without convergence: How political context shapes the adoption of regulatory impact assessment’, Journal of European Public Policy 12: 924-943.

Radaelli, C.M. (2008) What do governments get out of regulatory reform?, Paper delivered to the 15th

Radaelli, C.M. and B. Dente (1996) ‘Evaluation strategies and analysis of the policy process’, Evaluation, 2:1, 51–66.

conference of the Nordic Political Science Association, University of Tromsoe, 6-9 August.

Radaelli, C.M. and F. De Francesco (2007) Regulatory Quality in Europe: Concepts, Measures and Policy Processes, Manchester, Manchester University Press.

Radaelli, C.M., F. De Francesco and V.E. Troeger (2008) ‘The implementation of Regulatory Impact Assessment in Europe’, Paper presented at the European Network for Better Regulation, Workshop on Regulatory Quality: Developing tools, approaches and sources for research on impact assessment, University of Exeter, Exeter 27 and 28 March

Renda, A. (2006) Impact Assessment in the EU: the State of the Art and the Art of the State. Centre for Policy Studies, Brussels


27

Rose-Ackerman, S. (2007) ‘Public choice, public law and public policy’ Paper presented at the First World Meeting of the Public Choice Society, Amsterdam, March 31.

Thatcher, D. and M. Rein (2004) “Managing Value Conflict in Public Policy”, Governance, 17:4, 457-486

Tavistock Institute (2003) Evaluation of Socio-Economic Development – The Guide, London: Tavistock Institute, www.evalsed.info/frame_guide_intro.asp.

TEP [The Evaluation Partnership] (2007) Evaluation of the Commission's Impact Assessment System. Final Report Submitted to Secretariat-General of the European Commission (Contract Number SG-02/2006). April 2007.

Sabel, C.F. (1994) ‘Learning by monitoring: the institutions of economic development’, in N. Smelser and R. Swedberg (eds), Handbook of Economic Sociology, Princeton, NJ: Princeton University Press and Russell Sage Foundation, pp. 137–65.

Van de Walle, S. (2008a, forthcoming) ‘International comparisons of public sector performance: how to move ahead?’ Public Management Review.

Van de Walle, S. (2008b) ‘Comparing the performance of national public sectors: Conceptual problems’. International Journal of Productivity and Performance Management, 57 (4): 329-338.

Visaggio, M. (2007) ‘A conceptual framework toward composite indicators for evaluating RIA program quality’, Paper presented at the European Network for Better Regulation Workshop, available at www.enbr.org/public/ENBR%20WP%20082007.pdf

Wallsten S., G. Clarke, L. Haggarty, R. Kaneshiro, R. Noll, M. Shirley, and L.C. XU (2004) The Telecommunication and Electricity Regulation Database, AEI-Brookings Joint Center for Regulatory Studies Working Paper, No. 04-05, Washington DC: AEI-Brookings Joint Center for Regulatory Studies.

Williams, M.E. (2002) ‘Market Reforms, Technocrats, and Institutional Innovation,’ World Development, 30:3, 395-412.

World Bank (2002) World Development Report 2002: Building Institutions for Markets, Washington DC: The World Bank.


28


29

Appendix: Set of Indicators of Regulatory Quality

This appendix contains one level of background indicators (no.1 below) and five levels or layers of indicators that can be used to map out the quality of Investment Climate Department’s Projects. For each layer we have identified ten indicators. In several cases, the indicator refers to the overarching regulatory quality initiative, involving comprehensively four regulatory quality tools - simplification, RIA, consultation, and access (see glossary). When this is not the case, we identify the specific regulatory quality tool associated. We relate the indicators to the IC typology of better regulation projects, i.e. ‘regulatory costs’, ‘M&E tools’, ‘regulatory risks’, and ‘administrative procedure for regulatory quality’. The column is completed by information about what, how (specifying the type of measure used and data gathering), and why (providing an interpretation of the indicators) is measured.

The background indicators and the five layers are grouped as follows:

1. Indicators of the political-administrative context (background)

2. Better regulation policy indicators

3. Quality of organizational design

4. Guidance and support

5. Output indicators

6. Impact indicators

1. Indicators of the political-administrative context

These indicators cover the political-administrative context. The latter is a precondition for an effective adoption of better regulation policies. They tend to capture the overall administrative context and do not intend to be considered as measures of the quality of the better regulation project. Their aim is to map the approach and culture of rulemaking, taking into account due process and consultation.

As explained in the paper, projects aimed at adopting and implementing better regulation tools should take into account the administrative context of a specific countries. The selection of the most effective tool is also related to the typology of administrative process in place in a specific country. Accordingly, one may expect that better regulation policy tools and RIA are better implemented in a country with obligations to justify regulation that are well-established and pre-date RIA.

The following table specifies the name and the main justification for the selection of the indicator. The data collection has to be conducted through desk research at the beginning of the WB-IC project with the support of government officers.


30

Indicators

Regulatory reform area

What is measured How is measured Interpretation

Regulatory costs

M&E tools

Regulatory

Procedures for regulatory quality

1 Requirement to justify regulation

X X X X The existence of an administrative requirement for justifying the regulatory intervention. The requirement can be supplemented by explanatory notes published in the official gazette

Objective yes/no indicator: presence-absence of this administrative requirement. Data gathering: basic analysis of regulatory and legal processes

The presence of this administrative requirement is a pre-condition for dialogue and transparency. This is also an incentive to clarify goals for each proposal

2 Mandatory administrative standards for rulemaking process

X X X X The existence of a mandatory governmental policy or standards that discipline how regulators make new regulations

Objective yes/no indicator: presence-absence of rulemaking standards. Data gathering: basic analysis of regulatory and legal processes

The presence of such administrative framework facilitates the institutionalization of better regulation tools

3 Obligation to publish the preliminary analysis of proposed regulation for public comment Regulatory quality

X X The existence of an administrative requirement/government policy to publish a preliminary report on regulatory proposals allowing stakeholders to comment and send feedback


It enhances consultation and fosters legitimacy of rulemaking process. It fosters legitimacy of consultation processes and embeds these processes


31

Indicators



Regulatory costs

M&E tools

Regulatory


tool: Consultation and Access & transparency

4 Ex-post review of regulation (by courts, parliamentary committees, council of state) Regulatory quality tool: Access & transparency

X The regulatory process should have a stage of review of contents and process of new regulation. According to the legal origin and state tradition of a country, this review may be conducted by courts, parliamentary committees, or council of state


The presence of ex-post review, and clarity on the body that performs it, increases quality of rule-making

5 Existence of an official Handbook on sound policy analysis Regulatory quality tool: RIA

X X The existence of a government handbook on the economic appraisal of policy across its life-cycle (from adoption to evaluation) covering all policies in all departments and regulatory agencies. Ex: the UK Treasury green book

Objective yes/no indicator: presence-absence of an appraisal handbook with these characteristics. Data-gathering: basic analysis of official documentation

To know about the existence, the content and the degree of sophistication of the handbook provides information on the expectations of the government about the economic analysis of regulation


32

Indicators



Regulatory costs

M&E tools

Regulatory


6 Presence of appraisal tools in administrative law and policy procedures Regulatory quality tool: RIA

X X The existence of policy appraisal tools such as environmental impact assessment (EIA), sustainability appraisal, trade analysis, health impact assessment, sectoral risk analysis

A set of objective yes/no indicators of capacity that already exists in the country. Data to be gathered by surveying departments and key agencies. Indicators can be grouped in a [0,1] index

Knowing the existence, the content and the degree of sophistication of appraisal tools inserted in broader administrative processes provides information about the capacity to carry out economic analysis

7 Explicit policy (adopted by the government, cabinet or in law) promoting government-wide regulatory reform or regulatory quality improvement (i.e. better regulation policy)

X X X X The existence of a set of objectives, principles, and responsibility of who does what in better regulation policy

Objective yes/no indicator: presence-absence of an explicit regulatory quality policy. Data gathering: basic analysis of the governmental policy for high quality regulation and lawmaking (better regulation policy)

A program defined in terms of objectives, principles and responsibility is a pre-condition for efficient better regulation projects

8 Act(s) on freedom X X The existence of mandatory rules, Objective yes/no indicator: The existence of a freedom of


33

Indicators



Regulatory costs

M&E tools

Regulatory


of information Regulatory quality tool: Access & transparency

coordinated in a single legal document, for the access to information possessed by government offices

presence-absence of freedom information act(s)

information - access is a prerequisite of better regulation policy and increases the transparency of regulatory analysis

9 Performance management system

X X The implementation of an overall performance management system that monitors and evaluates departments and the performance of the higher civil service

Yes/no indicator: presence-absence of a performance management system relying on survey. The indicator is objective although it may require some subjective judgments about whether performance management covers all departments or few, it is merely adopted or also implemented, and so on.

The existence of an overall performance management system is a prerequisite of accountability and rationality. This system facilitates the introduction of incentives for delivering high-quality regulations

10 Legal origin of the country

X X X The type of legal of a country as defined by Andrei Shleifer’s data set www.economics.harvard.edu/faculty/s

Categorical indicator to be obtained from existing datasets

There is a vast economic literature on the impact of legal origin on economic growth.


34

Indicators



Regulatory costs

M&E tools

Regulatory


hleifer/dataset Dataset are also available in the CIA world facts book: https://www.cia.gov/library/publications/the-world-factbook/index.html

Knowing the type of legal origin of a country provides useful information of what type of regulatory reform can be pursued. It may be useful also for benchmarking countries within the same legal origin

2. Better regulation policy indicators

These indicators refer to the design of regulatory reform - better regulation policy. Better regulation projects should promote a sound policy composed of different elements. This set of indicators is aimed at supporting the project administrator in the design of the project. These indicators also facilitate the implementation and evaluation of better regulation policy. To illustrate, the presence of quantifiable targets is a necessary condition for an effective evaluation. The following table specifies the name and the main justification for the selection of each indicator. Data collection can be carried out (preferably) by government officials collecting official documents. Yes-No indicators can be operationalized as 1 (yes) and 0 (no); when we suggest composite measures, they tend to be indexes ranging from 0 to 1.

https://www.cia.gov/library/publications/the-world-factbook/index.html�

https://www.cia.gov/library/publications/the-world-factbook/index.html�


35

Indicators



Regulatory costs

M&E tools

Regulator


1 Regulatory reform objectives

X X X X Explicit objectives of regulatory reform embedded in the highest-level official documents (legislative program or coalition agreement) of the executive

Objective yes/no indicator: presence-absence of regulatory reform objectives relying on analysis of better regulation policy

The existence of regulatory reform objective within highest-level official document of the executive sets the objectives against which it will be evaluated later

2 Principles of regulatory quality

X X X X The existence of principles of regulatory quality. They should be set after having received an opinion from Parliament and open consultation with stakeholders

Objective yes/no indicator: presence-absence of principles of high quality regulation Data to be gathered via basic analysis of better regulation policy

The project should promote the establishment of regulatory quality principles through an open and wide discussion within the political and societal arenas

3 Objectives of regulatory quality

X X X X Explicit objectives of quality Objective yes/no indicator: presence-absence of regulatory quality objectives. Data to be gathered via an analysis of better regulation policy

In order to be measured, programs must contain objectives. This indicator makes the notion of regulatory quality explicit and measurable


36

Indicators



Regulatory costs

M&E tools

Regulator


4 Control capacity (Mandatory oversight affecting the major steps in the life-cycle of regulation)

X X X X The existence of a mandatory oversight of ten phases, that is, regulatory formulation, consultation, production, ex-ante analysis, access, enforcement, ex-post review, codification, simplification, and elimination. This requires the establishment of a central unit in charge of the overall control of regulatory governance

Objective set of yes/no indicators: presence-absence of mandatory oversight. Data gathering: analysis of the regulatory process. [0,1] index with values of 0.1 for each of the ten phases

This measure captures the control capacity of the executive

5 Ministerial accountability

X X X X The existence of an individual minister accountable for progress on better regulation against measurable benchmarks

Objective yes/no indicator: presence-absence of ministerial accountability concerning high quality regulation. Data to be gathered via official documents on the composition of government

Evidence of political commitment and accountability

6 Annual regulatory agenda debated in parliament

X The existence of an annual regulatory agenda debated in parliament and available on-line

Yes/no indicator: presence-absence of annual regulatory agenda. Data to be gathered via the

The project promote the establishment of an annual regulatory agenda to foster wide external accountability and


37

Indicators



Regulatory costs

M&E tools

Regulator


Regulatory quality tool: Consultation Access and transparency

examination of the regulatory process. It may require some subjective decisions on what exactly constitutes a regulatory agenda across countries with different political systems

secure political commitment of elected assemblies

7 Regulatory reform principles such as maximization of collective welfare

X X X X The existence of principles that target collective welfare or other widely shared principles of better regulation (“shared” by FIAS and the recipient country)

Objective yes/no indicator: presence-absence of an explicit and clearly stated principle Data gathering: official documentation and guidance published by the government

A program defined in terms of objectives, principles and responsibility is a pre-condition for efficient regulatory policies

8 Regulatory policy states that decisions be based on risk analysis Regulatory quality tool: RIA

X The existence of a principle that anchors decisions to risk analysis

Yes/no indicator: presence-absence of risk based regulatory quality. Data gathering: official documentation and guidance published by the government

The presence of risk analysis principle encourages explicit comparisons between risks


38

Indicators



Regulatory costs

M&E tools

Regulator


9 Quantifiable targets and measurable benchmarks Regulatory quality program in general but in particular Simplification

X X The existence of quantifiable targets and benchmarks in better regulation

Yes/no indicator: presence-absence of quantifiable targets. Data gathering: official documentation

Quantifiable targets facilitate the implementation, management and evaluation of better regulation

10 Department of economic analysis within core executive Regulatory quality tool: Simplification and RIA

X X X The existence of a department for economic analysis within the core executive

Yes/no indicator for the existence of a department of economic analysis. Data to be provided by a basic survey of departments

The presence of economists creates capacity for economic analysis of regulation


39

3. Indicators of organizational design

These indicators refer to the organizational dimension of better regulation policy. Their general aim is to map the extent of institutionalization of administrative innovations and better regulation policy and its tools. These indicators change the incentives within the administrative structure. They are also measures of quality assurance process and are essential for effective shift from adoption to an effective implementation.

The following table specifies the name and the main justification for the selection of the indicator. Data collection can be carried out by government officials collecting and analyzing official documents.

Indicators



Regulatory costs

M&E tools

Regulatory


1 Central (quality assurance) unit (CU) Sub-indexes: a) authority to oversee regulatory process b) number of staff c) number of staff in departmental or

X X X X The existence of a body (typically in the cabinet office) in charge of monitoring and quality assurance. The diffusion of quality assurance across departments and regulatory agencies

Set of objective of indicators: presence-absence and the extent of the central unit. Data to be provided via the examination of better regulation policy This set of indicators can be aggregated in a [0,1] index

Administrative body/bodies overseeing better regulation are necessary for efficient implementation, management and evaluation


40

Indicators



Regulatory costs

M&E tools

Regulatory


agency-level regulatory reform units

2 Board-level champions of regulatory reform in departments and/or agencies

X X X X Promotion of champions (top-level civil servant and experts) of regulatory reform

Objective yes/no indicator: presence-absence of a high-level unit engaged with better regulation

Top level civil servants and experts of regulatory reform legitimate the better regulation policy. They also enhance the communication policy of the government

3 Independent monitoring/evaluation of regulatory reform

X X X X The existence of a monitoring program and dedicated independent body

Objective yes/no indicator: presence-absence of monitoring program

The establishment of an administrative body (different from the central unit) in charge of monitoring the progress and results of better regulation fastens the institutionalization of better regulation policy

4 Public annual X X X X The existence of CU's commitment to Objective yes/no indicator: Reporting enhances the


41

Indicators



Regulatory costs

M&E tools

Regulatory


report of the CU

produce a report and make it public presence-absence of CU's annual report

accountability and transparency of the central unit. Additionally, it is useful for evaluating the policy

5 Public official documentation on strategy to involve citizens and stakeholders Regulatory quality program in general and Consultation and Access & transparency

X The existence of an official document explaining the strategy to involve citizens and stakeholders on regulatory reform

Objective yes/no indicator: presence-absence of a strategy to involve citizens and stakeholders on regulatory quality and reform initiatives

A strategy to involve citizens increases the legitimacy of better regulation programmes

6 Better regulation documents (annual reports, annual agenda, consultation and simplification strategies, RIAs)

X The existence of a portal where all consultation documents, main better regulation strategy documents and reports as well as RIAs are collected.

Objective Yes/No indicator: presence-absence of a portal for enhancing the access and transparency of regulatory quality programs

Websites facilitate the access to and transparency of, regulatory consultation, process and decision


42

Indicators



Regulatory costs

M&E tools

Regulatory


are available through a single easily accessible portal Regulatory quality tool: Consultation Access and transparency

7 Financial resources: yearly change or trend

X X X X Yearly change in expenditure for better regulation (or five-year trend)

Objective yes/no indicator of the trend of the amount of financial resources dedicated to better regulation. Values: 0, if budget t1 < t0; 1 otherwise

Budget provides financial resources for implementing tools

8 Peer review of the analysis contained in regulatory analysis Regulatory quality

X X X The existence of a requirement to expose major economic analysis to independent peer-review

Objective yes/no indicator: presence-absence of peer review mechanisms. Data from the examination of regulatory quality programs

Peer-review increases the robustness and credibility of economic analysis contained in M&E tools, simplification and RIA


43

Indicators



Regulatory costs

M&E tools

Regulatory


tool: Simplification and RIA

9 Ex-post evaluation of regulations in term of their capacity to meet objectives Regulatory quality program in general and Simplification

X X The existence in the guidance of procedures to set up monitoring and ex-post evaluation of regulations

Objective yes/no indicator for the existence of ex-post evaluation of regulatory outcome. Data from the examination of regulatory quality program

Clearly defined procedures facilitate the ex-post evaluation of regulations. The latter is necessary in order to have efficient regulatory management

10 Regulatory targets included in the public service performance management

X X X X The inclusion of better regulation activities in the overall performance management system used in the public sector

Yes/no indicator for the existence of a performance management system that includes regulatory quality measures

Evidence of the importance of better regulation. Better regulation is embedded in the overall management system used inI the public sector. Individuals achieving results are rewarded


44

4. Indicators for guidance and support

These indicators refer to the activities conducted by governments to enhance the quality of regulations. These activities focus on the drafting of guidance and support activities for regulators. The following table specifies the name, the main justification for the selection of the indicator and the data collection methodology.

Indicators



Regulatory costs

M&E tools

Regulatory risk


1 Guidance for

Simplification,

Consultation,

RIA, Access & transparency (code of plain language drafting)

X X X X The existence of a written guidance document for each of the major tool. Guidance is published, widely circulated, and reviewed on a regular basis

Presence/absence of guidelines for each tool.

High-quality better regulation tools are based on specific steps and methods. Guidance on this process is necessary as well as a clear illustration and explanation of the methods

2 Targeting - Explicit criteria for selection of items to be scrutinized via Simplification, RIA

X X X X This indicator measures whether tools are targeted or not. It measures the presence/absence in the guidance of the major tools of explicit criteria specifying when economic analysis and/or consultation is mandatory

Yes/no indicator for the existence of the criteria

It requires some subjective judgments

Selection is fundamental if regulatory tools are to be targeted.

The cases of exclusion have to be transparent


45

Indicators



Regulatory costs

M&E tools

Regulatory risk


3 Regulatory reform objectives (see indicator No. 3, layer 2) and targets (see indicator No. 9, layer 2) of regulatory reform embedded in performance management and career progression

X X X X The existence of a performance management system that covers regulatory process and rewards regulators delivering high quality regulations

Two yes/no indicators of the existence of monitoring program relying on analysis of and survey on regulatory process. Aggregate measure with values: 0, 0.5, 1

This indicator captures the change in the incentive structure, facilitating the reward of regulators delivering high quality regulations

4 Simplification, M&E tools, consultation, RIA, access & transparency training composite (weighted) indicator

X X X X The existence of periodic training for all policy-officers supervising the implementation of better regulation tools. The weighted index is composed of:

Data on training can be aggregated in a [0,1] index

Training is an essential element. Periodic training diffuses best practices and skills.

Note: The higher civil service is defined with different measures in the literature: it consists of some 3000 people in a country like the UK.


46

Indicators



Regulatory costs

M&E tools

Regulatory risk


5 Integrated teams working on RIA, EIA, general policy appraisal, SCM and consultation

X X X X Number of team meetings attended by officers working on different dimensions of policy appraisal

Objective composite measure of the presence and diffusion of integrated teams

Teams integrate and share prior experience with environmental policy appraisal and cost-benefit analysis of infrastructure with simplification, consultation and RIA analysts

6 Simplification, Consultation and RIA guidance, or an overall regulatory quality policy mandates:

a) notice-and-comment;

b) Contact name;

c) Publication of documents underlying the analysis carried out;

X X X The existence of an official document explaining the strategy to involve citizens and stakeholders

Aggregate indicator of four objective of yes-no sub-indexes. Values: 0, 0.25, 0.5, 0.75, and 1

This set of requirements facilitates the inclusion of business and citizens. It increases the transparency of regulatory process and the legitimacy of regulations


47

Indicators



Regulatory costs

M&E tools

Regulatory risk


d) explain how consultation has informed assessment

7 RIA guidance mandates - contains:

a) Proportionality of analysis

b) Policy objectives as measurable goals- outcomes

c) An analysis of the baseline (including explanations of how to appraise it)

d) Instructions of how to handle trade-offs

X X The existence in an RIA guidance of the most important elements for sound economic analysis

Aggregate indicator of six subjective yes-no sub-indexes. Values: 0, 0.16, 0.33, 0.5, 0.66, 0.83, and 1.

These are the minimum content elements for sound regulatory appraisal


48

Indicators



Regulatory costs

M&E tools

Regulatory risk


e) Instructions on when to conclude RIA with the choice of an option and what to do when this is not possible

f) Detailed explanations and instructions on quantification and monetization of costs and benefits

8 Consultation guidance provides:

a) timetable and planning

b) consultation methodology

c) mandatory report

X X The existence of the most important elements for a transparent consultation processes

In particular:

a) a consultation timetable and plans to allow affected parties to comment on the assessment of alternative options

b) an illustration of different

Aggregation of three objective yes-no measures. Values: 0, 0.33. 0.66. 1

These are the minimum content elements for a transparent regulatory consultation


49

Indicators



Regulatory costs

M&E tools

Regulatory risk


methodologies to use in the consultation process

c) a requirement to publish a final report on the consultation process summarizing: affected parties consulted; input of consultation; explanation on how consultation has been taken into account; justification where it has not done so

9 Simplification guidance/strategy mandates:

a) quantifiable targets

b) simplify existing regulation when introducing new regulation

c) public register

X X The existence of the most important elements of effective simplification.

In particular:

a) measurable targets

b) a commitment to simplify existing and overlapping regulation when proposing new regulatory requirements

c)a public register of regulatory provisions accessible on-line

Aggregation of three objective yes- measures. Values: 0, 0.33. 0.66. 1

a) Measurable targets facilitate evaluation and review of simplification programs

b) This action connects the ex-ante and ex-post stages of regulatory quality programs

c) An inventory (available on line) boosts access and transparency


50

Indicators



Regulatory costs

M&E tools

Regulatory risk


10 Access strategy mandates:

a) complaint procedures

b) documents available on-line through a single access portal

X The existence of the most important elements for ensuring an effective oversight of regulatory process

In particular:

a) specific procedures that enable citizens to complain about the difficulties they have experienced in complying with regulation

b) the commitment to making the documents available on-line through a single access portal

Aggregation of the two dimensions

a) it shows that the project fosters a commitment to enhancing access by making citizens’ voice heard

b) Single access portal to all documents supporting the regulatory process enhance access and the transparency of the regulatory process

5. Output indicators

These indicators refer to outcome achieved by the project and better regulation policy. Their aim is to capture the progress achieved with tools. These measures gauge the extent of activities carried out and establish the value-for-money of the project. There are several indicators that are intended to strengthen the overall management & control of the project and the wider regulatory reform policy.


51

The following table specifies the name, the main justification for the selection of the indicator and the data-gathering methodology.

Indicators



Regulatory costs

M&E tools

Regulatory


1 Cost/ Dedicated budget for each regulatory quality tools foreseen in the better regulation for growth program

X X X X Basic information on the cost of better regulation

Financial resources spent for the implementation of better regulation. Information gathered specifically during the implementation of the WB project. Relevant data for the evaluation of the WB project

The knowledge of the costs for each of the better regulation tools foreseen in the project provides accountability and evidence that the donor and the government have basic information and control of the project

2 Number of RIAs, number of consultations, and/or number of existing regulations simplified carried out in a given year is known

X X X X Knowledge of the number of regulations that are tested via RIA, number of regulatory consultations, number of laws/regulations simplified

Results achieved during the implementation of each tool of better regulation. Information gathered specifically during the implementation of the WB project.

The number of regulations affected by the better regulation project provide accountability and evidence that the donor and the government have basic information and control of the project


52

Indicators



Regulatory costs

M&E tools

Regulatory


Relevant data for the evaluation of the WB project

3 Percentage of proposals for which a RIA, simplification, and/or consultation were carried out is known

X X X X See above Continuous indicator of activities. Information gathered specifically during the implementation of the WB project. Relevant data for the evaluation of the WB project

See above

4 Annual report on total net costs of new regulations minus costs of regulations repealed in the current year

X X The annual report provides data on total regulators costs

Yes/no indicator of the existence of CU's annual report relying on the cost saved through simplification and M&E tool Information gathered

A detailed annual report on overall costs saved in a specific year increases the accountability of better regulation programs


53

Indicators



Regulatory costs

M&E tools

Regulatory


Regulatory quality tool: Simplification

specifically during the implementation of the WB project. Relevant data for the evaluation of the WB project. Examination of all simplification initiative conducted in a specific year

5 Percentage of RIAs that comply with guidance (re: indicators 7 in the previous table) Regulatory quality tool: RIA

X Using the indicators of the previous table, this ratio capture the overall quality of RIA

Continuous indicator of the knowledge of good quality RIAs. Information gathered specifically during the implementation of the WB project by experienced officers and researchers Relevant data for the evaluation of the WB project through an

The ratio enable the donor and the government to gauge the progress and the extent of learning


54

Indicators



Regulatory costs

M&E tools

Regulatory


examination of a sample of RIAs

6 Percentage of consultations that comply with guidance (re: indicator 8 in the previous table) Regulatory quality tool: Consultation

X This ratio captures the overall quality of consultation

Continuous indicator of the knowledge of good quality consultations. Information gathered specifically during the implementation of the WB project. Relevant data for the evaluation of the WB project through an examination of a sample of consultation

The ratio enable the donor and the government to gauge the progress and the extent of implementation

7 Percentage of simplification initiatives that comply with the overall strategy (re: indicator 9 in the

X X This ratio captures the overall quality of simplification strategy

Continuous indicator of the knowledge of good quality simplification. Information gathered specifically during the implementation of the WB

The ratio enable the donor and the government to gauge the progress and the extent of implementation


55

Indicators



Regulatory costs

M&E tools

Regulatory


previous table) Regulatory quality tool: Simplification

project. Relevant data for the evaluation of the WB project through an examination of a sample of simplification initiatives

8 Compliance with the access strategy (re: indicator 10 in the previous table) Regulatory quality tool: Access & transparency

X This ratio captures the overall quality of access strategy

Continuous indicator of the knowledge of good quality simplification. Information gathered specifically during the implementation of the WB project. Relevant data for the evaluation of the WB project through an examination of government’s publication and websites

The ratio enables the donor and the government to gauge the extent of implementation


56

Indicators



Regulatory costs

M&E tools

Regulatory


9 Online data, information and automatic document provision Regulatory quality tool: Access and transparency in particular e-registry

X X X This indicator captures the overall impact of simplification and access strategy (through e-registry).

Continuous indicator of the number of forms and guidelines for filling forms uploaded in a single direct government portal. This is a World Bank Business enabling environment project

This measure enables the donor and the government to gauge the effectiveness of implementation of access strategy

10 Reduction in the time it takes to file taxes Regulatory quality tool: Simplification Access and transparency in particular e-

X X X This indicator captures the overall impact of simplification and access strategy (through e-registry).

Continuous indicator of the number of business days saved by taxpayer in filling up tax forms. This is a World Bank Business enabling environment project

This measure enables the donor and the government to gauge the effectiveness of implementation of simplification and access strategy


57

Indicators



Regulatory costs

M&E tools

Regulatory


registry

6. Impact indicators

These indicators refer to impact of better regulation program. The measures selected are a mix of subjective and objective indicators. Several measures are provided by international organizations such as the United Nations and the World Bank. The aim of these indicators is to capture different dimensions of impact.

The following table specifies the name, the main justification for the selection of the indicator and data collection methodology.


58

Indicators



Regulatory costs

M&E tools

Regulatory


1 Economic analysis applied to the whole life-cycle of regulation and in competition policy

X X X X This is a measure of institutionalization of better regulation policy.

Subjective assessment by independent experts-researchers

Overall coherence of better regulation policy where its tools are based on the methodological concept of RIA

2 Survey-based indicators measuring the beliefs of regulators

X X X X Percentage of regulators who think that RIA:

a) reduces departmental conflict at the stage of policy formulation b) increases the likelihood that the final decision will be more likely to achieve policy goals c) facilitates communication between the senior civil service and the political level

Ad hoc representative survey every four years

Change in the regulators’ behavior The change has to be measured between one survey and the next.

3 Survey-based indicators gathered on firms and citizens

X X X X Examples: - % of firms which think that regulatory changes are more predictable than before - % of firms which think that the better regulation policy has an impact

Ad hoc representative survey every four years

Subjective indicators on the change in the quality of regulation as perceived by firms and citizens. Change to measured between one survey and the next.


59

Indicators



Regulatory costs

M&E tools

Regulatory


in opening markets, - % of citizens that feel that their administrative obligations have been reduced by… over the last…. Years - % of citizens who think that their views are taken on board in the development of regulatory proposals

4 Doing Business indicators and/or product market indicators for the OECD countries

X X X X The annual report on the project has to provide data on improvement with DB indicators

World Bank data These are composite and objective measures of the progress achieved with better regulation tools

5 Kauffmann et al.’s regulatory quality indicators

X X X X The annual report has to provide data on improvement with Kauffmann regulatory quality indicators

World Bank data These are composite and subjective measures of the progress achieved

6 Productivity of SMEs, industry-level multifactor productivity growth (for the

X X The annual report has to provide data on improvement with SMEs and industry-level productivity

World Bank data This composite measure enables governments to gauge the progress achieved with better regulation tools


60

Indicators



Regulatory costs

M&E tools

Regulatory


OECD countries)

7 Increase in foreign direct investment and rate of investment in high-growth sectors (such as telecommunica-tions and bio-technology) (summary innovation index for the EU member states)

X X X The annual report has to provide data on improvement with foreign direct investment

United Nations data http://www.unctad.org/Templates/StartPage.asp?intItemID=2921&lang=1

This simple measure enables governments to gauge the progress achieved with better regulation

8 Average official cost to comply with business regulation or variation in the net growth of start-ups for the EU member

X X The annual report has to provide data on reduction to comply with business regulation

This is a World Bank Business enabling environment project

This aggregated measure enables governments to gauge the progress achieved with simplification and better regulation


61

Indicators



Regulatory costs

M&E tools

Regulatory


states (data collected by Eurostat)

9 Net benefits as measured by total RIAs tn / net benefits tn-1 Regulatory quality tool: RIA

X Ratio of the improvement in delivering net regulatory benefits

Increase or decrease of net benefits delivered by RIA. Examination of the entire population of RIAs conducted in a specific year

This is a top simple measure that synthesizes the overall quality of RIA and better regulation policy

10 Number of lives saved by new regulations Regulatory quality tool: RIA

X X Overall measure that assesses the capacity of regulation to save lives

Continuous indicator of the number of lives saved by new regulation Examination of the entire population of RIAs conducted in a specific year

This measure synthesizes a fundamental dimension of quality. It can also be used in communication campaigns targeting citizens


62

GLOSSARY

BETTER REGULATION POLICY OR REGULATORY REFORM

Better regulation policy refers to a set of principles and tools that discipline the regulatory policy process. ‘Better regulation’ does not mean that the quality of the regulatory outcome targeted by the governments’ initiative result in higher quality. Better regulation as other public policies can be successful or fail. We consider better regulation as a public policy because it is formed and shaped by its own actors, problems, resources, rules of interaction, decision-making structures and outcomes. The following are the major policy instruments used within a better regulation policy.

RIA

RIA (Regulatory Impact Assessment) or simply Impact Assessment is a systematic and mandatory appraisal of how proposed primary and-or secondary legislation will affect certain categories of stakeholders, economic sectors, and the environment. “Systematic” means coherent and not episodic or random. “Mandatory” means that it is not a voluntary activity. Essentially, RIA is a type of administrative procedure, often used in the pre-legislative scrutiny of legislation.

Its complexity and analytic breadth vary, depending on the issues at stake and the resources available - the degree of complexity should be proportional to the salience and expected effects of the regulation. Indeed, the expected effects analyzed via RIA may cover administrative burdens or basic compliance costs, or more complex types of costs and benefits, including environmental benefits, distributional effects, and the impact on trade. The scope of economic activities covered by RIA ranges from some types of firms to whole economic sectors, competitiveness and the overall economic impact of regulations. RIA can also be used to appraise the effects of proposed regulations on public administration (e.g., other departments, schools, hospitals, prisons, universities) and sub-national governments. Although RIA is often used to estimate the impact of proposed regulation, it can be used to examine the effects of regulations that are currently in force, for example with the aim of eliminating some burdensome features of existing regulations or to choose the most effective way to simplify regulations.

Consultation

Consultation is the process of ascertaining the views and opinions of relevant experts, parties that would be affected by regulation and stakeholders. In RIA, it is also a means to ascertain the need for regulatory intervention and the likely costs and benefits arising out of different regulatory options.

BRG: Pro

63

Simplification

Administrative measures

These are measures that reduce administrative burdens (paperwork and informational requirements) imposed by governments on enterprises, citizens and the public sector. They include diverse instruments such as drafting in plain language, physical one-stop shops, simplification of licensing procedures, time limits established for decision-making, and the use of information technology to simplify dealings with administrations, such as e-government initiatives and web portals.

Standard Cost Model

‘The SCM [standard cost model] has been developed to provide a simplified, consistent method for estimating the administrative costs imposed on business by central government. It takes a pragmatic approach to measurement and provides estimates that are consistent across policy areas. The SCM methodology is an activity-based measurement of the businesses’ administrative burdens that makes it possible to follow up on the development of the administrative burdens.’ SCM network website www.administrative-burdens.com/default.asp?page=122

Doing Business Indicators

‘The indicators presented and analyzed in Doing Business measure business regulation and the protection of property rights -- and their effect on businesses, especially small and medium-size domestic firms.

First, the indicators document the degree of regulation, such as the number of procedures to start a business or to register and transfer commercial property. Second, they gauge regulatory outcomes, such as the time and cost to enforce a contract, go through bankruptcy or trade across borders. Third, they measure the extent of legal protections of property, for example, the protections of investors against looting by company directors or the range of assets that can be used as collateral according to secured transactions laws. Fourth, they measure the flexibility of employment regulation. Finally, a set of indicators documents the tax burden on businesses.’ (Doing business website: www.doingbusiness.org/AskQuestion/)

Non-administrative measures

These measures include, for example, review and improvement of the substance or legislative approach of existing regulation with the aim of reducing any regulatory burden imposed by governments on enterprises, citizens and the public sector. The reviews can be generalized, covering the entire body of the regulation, or more specific, concentrating on a type of regulation or sector.

Access and transparency

Access can be improved by practical arrangements (especially using information and communication technology). Transparency involves programs aimed at improving the understanding and use of international (WTO, European, etc.) and national regulation.

BRG: Pro

64

This leads to coherence and clarity of regulations. Examples: the existence of specific procedures that enable citizens to complain about the difficulties they have experienced in complying with regulation Commitment to making the documents available on-line through a single access portal.


65

Annex: Better Regulation for Growth Program

The Better Regulation for Growth (BRG) Program was launched in 2007 by the Dutch Ministry of Foreign Affairs, the UK Department for International Development (DFID) and FIAS, the investment climate advisory service of the World Bank Group.

The objective of the BRG is to improve the regulatory and investment climate in developing countries, thereby stimulating private sector investment, economic growth and poverty reduction. The BRG program aims to achieve this by developing and disseminating for the first time widely practical and operational guidance that will help developing countries design and implement effective regulatory reform programs. Regulatory governance reforms focus on the operation of complex and dynamic regulatory systems including regulatory policy development, administration and institutions. It is particularly relevant to developing countries where there is a clear need to improve regulatory systems. Regulatory reforms are most effective when linked to other government priorities, when certain preconditions are in place and when the design of the reform program has clear objectives and is context specific. Successful regulatory reform programs typically include a mix of components including one-time reductions in existing business costs and also regulatory governance reforms to policy making, administration and institutions which make it easier to do business. Regulatory governance reforms also results in more effective regulation by reducing risks for business and the broader community, improving resource allocation and building regulatory frameworks for socially beneficial and sustainable growth. The BRG Program has resulted in preparation of eight policy papers on regulatory governance issues, covering a broad spectrum: from regulatory governance, links to competition policy, regulatory institutions and tools to indicators for regulatory quality. It has also involved preparation of five country case studies on regulatory capacities in selected African countries. The web portal www.ifc.org/brg is part of the BRG Program and contains key documents, including references extracted from a comprehensive compendium of resources on regulatory management and reform and a newly developed Regulatory Impact Analysis (RIA) database.

http://www.ifc.org/brg�

Better Regulation for Growth - World Bankdocuments.worldbank.org › curated › en › ...Better...

Documents

Transcript of Better Regulation for Growth - World Bankdocuments.worldbank.org › curated › en › ...Better...