Regulations: To Evaluate or Not to Evaluate?

Catherine DymondStacey PrieurJune 10, 2013

2

• International Context• Domestic Context• CIC Regulatory Context• CIC Challenges• Recent Policy Change• So Now What?

Introduction and Overview

3

• New question in the international regulatory community: how do we evaluate regulations?

• Organization for Economic Co-operation and Development (OECD): intergovernmental organization that develops best practices for member states

• Canada chaired OECD’s Regulatory Policy Committee in 2012– Canada was key thought leader, and seen by many as paving

the way in practice, on the subject of how to monitor the performance of regulations and how they should be evaluated

International Context

4

• In 2012, the OECD’s Council on Regulatory Policy and Governance adopted 12 key recommendations, which included recommendation #5:

“Conduct systematic programme reviews of the stock of significant regulation against clearly defined policy goals, including consideration of costs and benefits, to ensure that regulations remain up to date, cost justified, cost effective and consistent, and deliver the intended policy objectives.” (emphasis added)

International Context

5

• Between 1975 and 1999, the Government of Canada enacted approximately 25,000 regulations

• Deregulation and regulatory quality became a policy goal in Canada in the 1980s

• Government of Canada Regulatory Policy 1992, 1995, 1999– Focus on smaller, simpler, more efficient/effective regulation– Some mention of monitoring impacts by TBS and departments

• Cabinet Directive on Streamlining Regulation (2007)– Focus on maximizing net benefits to society and exploring

alternatives to regulation, through cost-benefit analysis– Performance Measurement and Evaluation Plan (PMEP)

required for high impact regulations

Domestic Context

6

PMS versus PMEPPerformance Measurement Strategy Performance Measurement and

Evaluation PlanPolicy / Directive

TBS Policy on Evaluation;Directive on the Evaluation Function

Cabinet Directive on Streamlining Regulation

Policy Authority Centre of Expertise for Evaluation (CEE), TBS

Centre of Regulatory Expertise (CORE), TBS

Purpose Ensure credible and reliable performance information is available to monitor and assess results of programs

Ensure that regulatory activities meet initial policy objectives and are accordingly renewed on an ongoing basis

Requirement All programs, including transfer payment programs, have a PMS

When the proposal is triaged as “high” impact

Evaluation responsibility

Heads of Evaluation responsible for "reviewing and providing advice on PMSs”

Heads of Evaluation “must review the PMEP”

Components •Program Profile•Logic Model

• Performance Measurement Strategy Framework•Evaluation Strategy

• Overview of Regulatory Proposal• Logic Model• Performance Measurement Strategy

Framework• Evaluation Strategy

7

• CIC is governed by two key Acts and 14 Regulations– Immigration and Refugee Protection Act– Citizenship Act

• Ways in which CIC uses regulation – Change conditions (e.g., for applying, appealing) – Legal enabler (e.g., to provide authority)– Change behaviour (e.g., to deter)

CIC Regulatory Context

8

• 6 PMEPs completed to-date– Visa Requirements for Mexican Nationals: changed the

conditions under which individuals could enter Canada– Biometrics: provided legal authority to collect fingerprints on

temporary residents– Refugee Reform “Designated Countries of Origin”: Changed

processing rules for individuals from typically non-refugee producing countries to deter them from claiming asylum in Canada

• 1 PMEP was underway

CIC involvement in PMEPs

9

• Nature of CIC regulations– How do you monitor performance of a legal authority?– How do you separate the impact of the regulations from a

larger program or initiative?• Duplication of effort (PMS versus PMEP)

– Why the need for 2 documents? Who does the reporting?– Drains resources in program areas and with stakeholders

• Policy Gap between CDSR and Evaluation Policy (2009)– CDSR points to Evaluation Policy– Evaluation Policy does not provide details for how to evaluate

regulations

CIC Challenges in Meeting the CDSR

10

• According to CORE, it has received just over 30 PMEPs from seven different departments

• Evaluation divisions have not always been involved in the development of the PMEP– Why has Evaluation not been involved?

Other Departmental Involvement in PMEPs

11

• Cabinet Directive on Regulatory Management (2012)– Focus on the life-cycle approach to regulatory management– No mention of a PMEP requirement

• Identify the intended results, ensure that monitoring and reporting activities are effective, integrate performance measures, collect performance information (s. 44)

– TBS still required a PMEP for a high-impact regulatory proposal

• Recent decision by TBS no longer requires a PMEP for high impact proposals – Unclear specific rationale for change in policy direction

Most Recent Policy Context

12

1. Do we need to have a specific focus on performance measurement and evaluation for regulations?

2. If yes, how do we do this and how do we add the most value?– Measurement (PMS, PMEP, RIAS, DPR)– Evaluation (What does the evaluation focus on? How do you

incorporate it into existing evaluation?)

3. If no, what is the risk of not doing it?

4. What kind of support /direction do we need from TBS (guidance, policy instruction, etc.)?

So Now What – As Evaluators do We Care?