Portland, Oregon Development for Professional Regulators ... · have the capacity to influence...

Two Approaches to Evidence-Based PolicyDevelopment for Professional Regulators

Portland, OregonSeptember 16, 2016

Council on Licensure, Enforcementand Regulation 2016 Annual Educational Conference 1

TWO APPROACHES TOEVIDENCE-BASED POLICYDEVELOPMENT FORPROFESSION REGULATORS

Evidence-Based Policy Making at the Collegeof Physicians and Surgeons of Ontario

Presentation Overview

• What is evidence?

• Why use evidence?

• CPSO Policy Review Process

• Types of evidence

• Case study

• Insufficient evidence – what to do?




Evidence-Based Policy

Evidence-basedEvidence-basedEvidence-based Evidence-informedEvidence-informedEvidence-informed

What is evidence?

Anything that can be used to bolster aparticular policy position.

Why Use Evidence?

Now what is theCollege making me

do?

I am trying to takecare of patients –do they just make

stuff up?




Why Use Evidence?

Why Use Evidence?

CPSO Policy Review Process

STEP 1: Research andPreliminary Consultation

STEP 1: Research andPreliminary Consultation





STEP 2: Analysisand Drafting

STEP 2: Analysisand Drafting


STEP 3:Consultation

STEP 3:Consultation

STEP 4: Re-draft and FinalizeSTEP 4: Re-draft and Finalize




What Types of Evidence• Research

Jurisdictional ReviewJurisdictional Review Legal /Case Law

Legal /Case Law

Literature ReviewLiterature Review

What Types of Evidence

• Consultation

– External

– Internal

• Public Opinion

Polling

Case Study• Physicians’ Relations with Industry: Practice,

Education and Research Policy




– Focus on 2 specific positions set out in thepolicy

• Receipt of Gifts byPhysicians

• Product Information /Marketing

What is the Evidence – Gifts?

Current College of Physicians and Surgeonsof Alberta standard:

“A physician must not accept any personal gift ofany monetary or other value from industry.”





Jurisdictional Review




Jurisdictional Review• Association of research-based

pharmaceutical companies (Rx&D)…

Rx&D Code of Ethics:

"Members must not offer toany Stakeholder... any gift – incash or in kind, or anypromotional aid, price,reward, or any other item asan incentive or reward..."

Rx&D Code of Ethics:

"Members must not offer toany Stakeholder... any gift – incash or in kind, or anypromotional aid, price,reward, or any other item asan incentive or reward..."

Includes common, low-value items such asagendas, mouse pads,stationary, etc…

Literature Review

Literature Review

University of PennsylvaniaScholarly Commons -Centre for Bioethics Papers(2003)

• Industry gifts of minimal valuehave the capacity to influencephysician behaviour, evenunconsciously.

• “Policies and guidelines thatrely on arbitrary value limits forgift giving or receipt of giftsshould be re-evaluated.”








Literature Review• Int. J. Radiation Oncology Biol. Phys. (2004)

• 96% of physicians admittedaccepting gifts.

• 5% agreed that they wereinfluenced.

• 33% agreed that otherphysicians were influenced.







Many Physiciansbelieve they are

immune to influence

Literature ReviewFamily Medicine (2012) JGIM (2011)

• Patients less likely totake a prescribedmedication ifphysician had recentlyaccepted a gift inreturn for listening toa pharmaceuticalrepresentative’spresentation.

• Patients whobelieve giftrelationships existreport lower levelsof physician trust &higher rates ofhealth care systemdistrust.



Accepting gifts risksundermining patient trust



Literature Review Conclusions:




What did We Put in the Policy?

New policy states:

“Physicians must not acceptpersonal gifts of any valuefrom industry or industryrepresentatives.”

What is the Evidence – ProductInformation / Marketing?

Literature Review• PLoS Med. (2010) • Studies of exposure to

information provided bypharmaceutical companieshad associations with higherprescribing frequency, highercosts, or lower prescribingquality or found no significantassociations.

• Authors found no evidence ofnet improvements inprescribing.




Literature Review• Journal of General and Internal Medicine

(2013)

• Serious adverse events were mentioned in 5-6%of promotions, although 45% were for drugswith FDA "black box” warnings of serious risks.

• Physicians judged the quality of scientificinformation to be good or excellent in 54% ofpromotions and indicated a readiness toprescribe 64% of the time.

• Serious adverse events were mentioned in 5-6%of promotions, although 45% were for drugswith FDA "black box” warnings of serious risks.

• Physicians judged the quality of scientificinformation to be good or excellent in 54% ofpromotions and indicated a readiness toprescribe 64% of the time.

What Did We Put in the Policy?

• New expectation around industry producedproduct information:

“Physicians must ensure that they criticallyevaluate any information provided by industryrepresentatives and do not solely rely on thisinformation when making clinical decisions.”

What if We Don’t Have Good Evidence?

Court Rulings

Patient and Physician Advocacy

Federal Legislation




Provisions in Marijuana forMedical Purposes Policy

• “While conclusive evidence regarding thesafety and effectiveness of dried marijuanaas a medical treatment is limited…”.

• “These expectations are grounded in theprinciples of medical professionalism set outin the Practice Guide, and take into accountthe best available evidence…”.

Thinking With Your Heart:Using Qualitative Evidence in

Policy Development @PEO

32

How Might We....? (Problem Statement)

• Increase % of online licence renewals?

• Improve customer service (in _____process)?

• Improve efficacy of CPD requirements?

• Improve practitioners’ ethical behaviour?

• Reduce #, types of complaints?

• Identify practitioner difficulties earlier?

• Think & operate more holistically?




Research on Regulatory Impacts

“...the most notable finding to emerge from this review isthe shortage of systematic knowledge on the mainresearch question...how does professional regulationaffect the behaviour of those subject to regulation?...”

Professional Standards Authority (UK)Rethinking Regulation, August 2015, p.6

34

What do we want to know?

Gain insight into perspectives, opinions,reasons of various users on

Regulatory tools & resources

Processes: Admissions, Renewals, Standards,CPD/QA, Complaints

Policies & Outcomes

35

Correlations vs. Causality

36

Source: “Spurious Correlations” website (www.tylervigen.com)




Quantitative

evidence

HOW THINGSHAPPEN

Qualitative

evidence

WHY THINGSHAPPEN

ThoughtsIdeas

Values, BeliefsMotivations

Opinions

Qualitative ToolsPROACTIVE

• Environmental Scanning• Strategic Planning• White Papers• Surveys/interviews• Social Media/Crowdsourcing• Focus Groups/Town Hall

Meetings• Advisory Boards/Councils• Communities of Interest/

Practice• Behavioural Insights/

”Nudging”• Human-Centred Design

REACTIVE• Stakeholder Feedback/

Consultation• Customer Service Audits• Compliance Audits• Complaints/Criticism• Respond to news stories/legal

cases

Behavioural Economics 101• People are irrational

decision-makers: “gut feel”,priming, habit, laziness,herding, biases, incompleteinformation, and otherinfluences (emotionalmemory, loss aversion, etc.)

• People can be ‘nudged’towards more “rational”decisions by understandingtheir feelings, values, andmotivations




“Take the Stairs instead” –4 Nudge approaches

Germany Japan SwedenSingapore

What Is Human-centred Design(Aka Design Thinking)?

HCD6 Key

Features

HCD6 Key

Features

Human-FocusedHuman-Focused

QualitativeQualitative

CollaborativeCollaborative

IterativeIterative

InteractiveInteractive

Visual/

Experiential

Visual/

Experiential




Regulatory Policy ModelsSystem-centred vs. Human-centred

Practitioner-Client/

Patient

Leg./Reg./Bylaw

Pro

cess

es

ITSyste

ms

Org. Structure

LicenceHolder/

Registrant

PRACTITIONER

CLIENT/

PATIENT/

FAMILY

44

Practitioner-Client RelationshipTECHNICAL INTERACTIONS

NON-TECHNICAL INTERACTIONS

Assessment/diagnosis/scope

Contract/treatment plan

Treatment/service/payment Feedback/Evaluation

Trust/Understanding

Compliance Expectations/Satisfaction

Communication/Dialogue/Feedback

Results/adjustments

Advertising/Choice

Safety/Legalrequirements

Comfort/reassurance

PRACTITIONER

TECHNICAL ZONE

Codes/Standards

PLI Provider

Builders/manufacturers

Suppliers (materials,software)

CLIENT/PATIENT/FAMILY

REGULATORY ZONE

Regulatory Body

Ministries, departments

Special Purpose bodies

Agencies, boards, commissions

PRACTICE ZONE

Employer

Consultants &Contractors

Allied Practitioners

FinancialEND

USERSWorkers

Public

45

GenericPractitioner’s

Ecosystem




One Size Does Not Fit All• Sole Practitioner

• Small/Medium/LargePractice

• Generalist/Specialist

• InterdisciplinaryFacility/Clinic

• Management

• Non-practising

46

Other Variables

• Location - Urban/rural

• Local/cross-state/global practice

• Age

• Gender

• # Years of Practice

• Domestic/foreign trained

• Adaptability to new technology

47

Typical Design ThinkingProcess

48

Personas

CustomerJourney

Maps




Ethnography – for empathy• Recorded fieldwork

on user experience by

– Observing/Shadowing

– “Walking in theirshoes”/simulation

– Active Listening

– Stories

– Interviews

Regulatory Service Channels Licensure (application)

Registration/inter-provincial/state transfers

Licence Renewal & Annual Reporting

CPD/QA

Professional Liability Insurance/Amenities

Practice Audits

Practice Standards/Advisory

Complaints & Discipline

Fitness to Practice

Elections

Practitioner-Centred Research (PCR)Project Purpose

• Through qualitative research, gain a deeperunderstanding of:– behaviours, attitudes, trends and challenges faced by engineers

offering service to the public throughout the province;

– efficacy of Professional Affairs tools; and

– values and drivers that motivate engineers in their work

• Identify how to improve Professional Affairstools and services and work towards a moreperson-centred and proactive PolicyDevelopment and delivery

51




Practitioner Centred Research Project | Phase 2: Practitioner Research | Final Report | May 27, 2016 52

RESEARCHSET-UP

Recruitment

Interview guide

Motivationsdeck

Screening plan

RESEARCH

12 interviews(recorded)

Transcripts frominterview audio

RESEARCH PROCESS: OVERVIEW

Transcriptsnippet sorting

12 practitionerstories

SYNTHESIS I

Themes fromtranscriptsnippets

Categories and

insightsclustered from

themes

SYNTHESISWORKSHOP

Profile patterns

Distill keyresearchfindings

Articulate 5personas

RESEARCH&

SYNTHESIS II

Validationsurvey: develop

and deploy

Understand keyresearchfindings

Identify keyopportunities

OPPORTUNITYWORKSHOP

Map potentialactivities

Personaposters

Summaryreport

RESEARCHPACKAGE

Next stepsplanning

Sample Engineer Persona

Problem Definition




Practitioner Centred Research Project | Phase 2: Practitioner Research | Final Report | May 27, 2016

Support employers toimprove their internal

training?

Leverage vastexperience to share

these experienceswith younger

audiences?

Provide practitionerswith technical

support?

Improve the public’sperception of

engineers?

Provide engineerstechnical resources?

Incorporate externalchange agents to help

current?

Provide engineersclarifications of

differences? (stds v.guidelines)

Get universities tounderstand what skills

and knowledgepractitioners really

need?

Inform engineersabout legalities all

regulatoryadministrative

structures (basics of

authority)?

Capitalize on theseprocesses to share

the knowledge base?(relying on peers,

colleagues, etc.)

Work with employersto identify entry level

job skills?

Determine engineersare valued in society?

Expand and improveguidance in ethical

issues?

Better explain thedifferences on the

website?

Obtain information onthe guidelines that are

being used?

Educate engineers toclarify PEO’s

mandate and role ofother organizations?

Engage with newlylicensed engineers to

reorient therelationship?

Inform practitionersabout organizations

that are moreappropriate for these

activities?

Improve PEO’swebsite?

Inform members ofchanging/overlapping

regulations?

Improve users

exposure to PEO

resources?

Change PAResources to make

them more accessibleto engineers?

Guide P. Engineers intraining and personal

development?

Promote regs andcodes and PA

Resources toengineers?

Help engineers tobetter understand

practice advisoryservices?

Evaluate the mostuseful formats for

learning?

Ensure youngerengineers know and

make use of practiceadvisory?

Clarify with PEO andOSPE about whose

role it is to promote &provide such

resources/ programs?

Discover theguidelines and topics

that engineers wouldfind useful?

Deal with youngengineers who are no

longer trained by theircompanies?

Better identifyemerging trends and

factors leading to newguidance?

Better targetcommunications to

promote responseand relevancy?

Connect more closelywith practitioners

above annual feepayment?

Create moreengagement between

engineers and PEO?

Educate council tofocus on regulatory

issues?

LOWIMPACT

HIGHIMPACT

How might we...

LOW EFFORT

HIGH EFFORT

IMPACT/EFFORT MATRIX CHALLENGESPotential challenges to take on as determined by the teamduring the Opportunity Finding workshop May 11, 2016

55

Professional Affairs (PA) Transformation

Overall Goal: Relationship between professional

engineers and Professional Affairs is more supportiveand collaborative

Issues to address via structuralrefinements:1. How to improve the navigation of PA services

2. How to increase awareness of PA service

3. How to make practice guidelines more useful

56

Professional Affairs (PA) Transformation

Overall Goal: Relationship between professional

engineers and Professional Affairs is more supportiveand collaborative

Issues to address through messaging &content:1. Engineers find it challenging to stay current with shifting and

overlapping regulations

2. Engineers find non-technical activities the most challenging intheir current roles

57




Reflections• very useful and eye-opening to have an

‘out- of-regulatory-body’ experience toget a more realistic, ground-levelperspective

• can be a humbling experience

• could be part of a different regulator-practitioner relationship, transformingthe organization

• challenge is how to continue andoptimize channels of dialogue 58

Evidence-based Culture Supports

• Critical importance of CEO/Board support &leadership – value of evidence-based decisions

• Gather resources and distribute/presentexamples of evidence-based policy (from othercolleges or government) if possible

• Audit your organization’s EBP readiness

• Introduce an evidence requirement intodecision documents

59

[email protected]

[email protected]




BONUS PACKAGE:ADDITIONAL RESOURCES

How EBP-Ready Are We?

All PD evidence-informed, we regularly dialoguewith stakeholders; include impact costs, andconsider alternatives to regulation

Some use of evidence, where convenient or

easily available; we talk with stakeholders at somepoint in the process

Evidence is rarely used, and usually after-the -factwhen solutions are proposed; we talk to

stakeholders to confirm direction at the end

What’s evidence?

62

Moving To An Evidence-informedCultureStaged approach

– Identify current data collection and researchresources; share data within org

– Start generating policy questions – what would youlike to know/understand better?

– Data analysis – risk factors > implications

– Pilot studies on policy initiatives to test/demonstrate value

– Baseline quantitative/qualitative data

– Discuss/promote/learning cycles

63




Selected Evidence-Based Policy Resources

• www.arnoldfoundation.org

• www.whatworkscities.bloomberg.org

• www.mitacs.ca

• www.horizons.gc.ca

• www.ahrq.gov/research/findings/evidence-based-reports

• www.sunlightfoundation.com

• www.results4america.org

Behavioural Insights UnitsWorldwide

• White House – social behavioral scienceteam

• Canada – Privy Council Office – CentralInnovation Hub

• Ontario – Treasury Board Secretariat

• UK – Cabinet Office (partnership)

• Australia – New South Wales – Premier &Cabinet

• World Bank – Global Insights Initiative

Evidence-based/informed policy –University Research Centres

• www.ohsu.edu/xd/research/centers-institutes/evidence-based-policy-center

• www.healthpolicyinstitute.pitt.edu/evidence-based-policy

• www.govlab.hks.harvard.edu/

• www.govex.jhu.edu/

• www.mowatcentre.ca

• www.inside.rotman.utoronto.ca/behaviouraleconomicsinaction




Selected DT ResourcesArticles:

• Design Thinking for Regulatory Policy, Parts 1 & 2 JordanMax, Engineering Dimensions, Jan/Feb., March/April 2016

• The Evolution of Design Thinking Harvard Business Review,September 2015

Reports:

• Design Thinking for Public Service Excellence, UNDP GlobalCentre for Public Service Excellence, 2014, Singapore

• Licence to Innovate: How Government Can Reward RiskWorking Paper 24, Institute for Competitiveness &Prosperity, Toronto, February 2016

67

Selected DT ResourcesBooks:

• Change by Design by Tim Brown

• A More Beautiful Question by Warren Berger

• Creative Confidence by Tom Kelley

68

Selected DT ResourcesWebsites and Blogs:• www.ideou.com, www.ideo.com (tools, case studies)

• www.dmi.org (articles, videos, slides)

• FastCo Design Thinking Blog

• Gov.uk Open Policy Making Toolkit (tools, case studies)

• MindLab (tools, case studies)

• Public Policy Lab (NYC)

• DIY Tool Kit (Nesta)(tools)

• Public Sector Innovation Toolkit (Gov't. of Australia)

69




US Federal Government DT Initiatives

• Lab@OPM

• HHS IDEA LAB

• DVA Center for Innovation

• USAID Global Development Lab

• US Digital Services

Portland, Oregon Development for Professional Regulators ... · have the capacity to influence...

Documents

Transcript of Portland, Oregon Development for Professional Regulators ... · have the capacity to influence...