Workshop Topics

Improving Healthcare Worldwide.

Assessing Needs in CME and CPD

Optimizing Live CME and CPD

Technology in CME and CPD

Using a Curricular Approach in CME and CPD

Who Am I?- SVP (~16 yrs in CME overall)Prova: >500,000 hours of CME total

~70,000 hours of CME annually

- Surveyor, Workshop Leader

- Fellow, Past MECCA Leader, ACWG

- Clinical Instructor – Center for Learning and Innovation/Emergency Medicine

- Instructor, Healthcare CommunicationsSchools of Medicine, Public Health and Journalism

- Contributing Author

- Host – Lifelong Learning

Workshop Rules

• Participation is encouraged and required!

• We will blend small group discussions and full group interaction

• You are a part of the faculty!

• I have some slides for each section– A positive outcome may be not having

to use them

First Question

How do YOU define CME?

Second Question

Do you think that CME differs around the world?

Third Question

What do you hope to get out of this workshop?

First – An Overview

The Current CME Environment - Global

• The need exists– Lifelong learning commitment of MDs

• The learners are there

• No consistency intra-country regarding– Regulations– Guidelines– Physician requirements

• Funding varies

The Current CME Environment - Global

• Lack of accreditation standardization

• Barriers– Overcoming time zone challenges– Language disparities– Varying treatment availability– Varying skill and sophistication levels– Ability for changes to be implemented

The Current CME Environment in the US

• CME too focused on individual conditions and not realistic patient presentation

– Co-morbidities covered broadly

• Most data-intensive with little appropriate interactivity

– Lack of appropriate use of Adult Learning Principles

• Little if any time course management

– Learning objectives without changes from year to year – WHY?

What we see in CME is a Scatter Pattern

NO STRATEGY OR LINKAGE!!!

Reinvention of the wheel over and over...

What we SHOULD see is a Pattern Driving to Strategic Approach

Needs Assessment Continuous Assessment Outcomes

Linking activities to curriculum approach based on Needs Assessment

Mixed media based on learners’ preferences, cognizant of distribution opportunities

Adaptation over time

“The endis theBeginning”

The 4 C’s Have Been Missing!

• Collaboration

• Collegiality

• Coordination

• Consistency

Section 1 – Needs Assessments

Section 1 – Needs Assessments

Question: How Do You Currently Assess The CME/CPD Needs of

Learners?

Section 1 – Needs Assessments

Question: What More Could Be Done To Assess CME/CPD Needs?

Needs Assessment “Best Practices”*

• Elements and qualities– Multi-faceted– Uses frameworks or theory– Exploring various perspectives– Use of objective and subjective data– Primary and secondary sources of information– Identified educable gaps

• “Needs Assessment” means different things to different people – so “Best Practices” vary– And that’s OK!

*Note: this is not exhaustive list!

The Needs Assessment

• Must go far beyond literature review • Validation of educational methodology• Rationale for recommendation of tactic(s)• Don’t just ask academics/KOLs• Look for geographic needs and variations• Assess learning preferences

– Channel preferences are key– Real-time vs. archived

What Should a Needs Assessment Contain?

• Educational gaps

• Disparities

• Learning style preferences

• Where the learners are or where they go for education

• Geographic variations

• Clinical and non-clinical information

• Competitive CME landscape

What Should a Needs Assessment Contain? - II

• Are other activities working?– Why or why not?

• More than just KOL opinions

• Who should teach

• Who should not teach

• Who needs to learn and how

• When should the activity take place

The Components

• Evidence– Literature– Interviews– Evaluations– Medical records

• Preferences– Media– Modalities – Channels

• Environment– Competitive activities– Non-competitive activities

Survey Finding: Clinical Decision-Making

14%

27% 29% 30%

23% 23% 21%

33%

0%

10%

20%

30%

40%

No Tx SAB LAB ICS

Guideline User Guideline Non-UserP = 0.02

Foster JA, et al. Med Gen Med. 2007;9(3):24

PCPs most commonly chose an inhaled corticosteroid as initial therapy for a patient with mild COPD. This choice that would be guideline consistent for asthma, but not COPD, suggesting confusion about key differences between these 2 common lung diseases.

PCPs who are guideline users appear more likely to offer treatment and utilize long-acting bronchodilators for patients with mild COPD.

Initial Pharmacotherapy for Mild COPD

Geographic Analysis

*Data presented at ATS Annual meeting May 24, 2008

Preferences of Live Meetings By Age Group

2% 5% 6%3%

12%

17%9%

5%

22%13%

9%

8%

22%

13%

8%

13%

42%

53%

67%72%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Less than 10years

11-20 years 21-30 years Greater than30 years

Dinner meeting

National meeting

Grand Rounds

Regionalmeeting

Annualconference

Local dinner meetings preferred by oldest group (72%)

In a Setting Where Needs Have Changed OR in a Serial Learning Environment

• Broader objectives can still apply• May need to be adjusted accordingly

– Primarily in the case of multiple activities

– Ongoing assessments re-evaluation of needs updating of objectives

Section 2 – Optimizing Live CME/CPD

Section 2 – Optimizing Live CME/CPD

Question: Is All Live CME/CPD The Same?

Live CME – The Location

• Global

• International

• National

• Regional

• Local

Approach to education in each environment is different

Live Example: From Guideline to Practice™

• Education dedicated exclusively to current and new guidelines

• Multi-supported, multi-topic, regional CME series

• Learners explore critical recommendations for improving patient care in the primary care setting

• Interactive and engaging series addressing a multitude of challenges and practice barriers

Interact With The Audience

• Validate needs assessments

• Pre-activity questions– Competence– Confidence

• Use audience response technology appropriately

• Post-activity questions– Repeat questions and show change

Mix Audiences

• Collaborative Forum– Care team approach

• Oncology example – Tumor Board model

– Multiple specialists• ACS example – ED, Interventional

Cardiology, Clinical Cardiology

• NSCLC example – Oncology, Pathology

• Use audience response to measure– Concordance and discordance

Section 2 – Optimizing Live CME/CPD

Question: How Are You Enhancing The Learning Environment in Live

CME/CPD?

Section 2 – Optimizing Live CME/CPD

Question: How Are You Measuring Outcomes in Live CME/CPD?

Outcomes Measurement Techniques

• Evaluation/Surveys

– Standardization

• Live meeting example(s)

– Pre- and post-activity questioning

– Post-activity surveys

• Case-control matching

– Live activities

– Enduring materials

– eCME

– mCME

• Patient-level data evaluation

35

Rethinking Outcomes

• Participation: how many attended?• Satisfaction: did they like it?• Knowledge: did anybody learn?

– Declarative knowledge (knows what should be done)– Procedural knowledge (knows how it should be done)

• Competence: can anybody do what they learned?– Shows how to do it in an educational setting

• Performance: did behavior change?– Actually does it in practice

• Patient health: did it improve?• Population health: did it improve?

Courtesy of Don Moore, PhD

We Measure Performance-based CME

Measuring Change – Outcomes Levels

Levels of Outcomes-based CME Evaluation Model*

Level Outcome Definition

1

2

3

4

5

6

7

Participation

Satisfaction

Learning

Competence

Performance

Patient Health

Population Health

Number of physicians/others who registered and attended

Degree to which participant expectations about the setting/delivery of CME activity were met

Changes in knowledge, skills, and/or attitudes of the participants

Changes knowledge, skills and behavior utilized to improve performance

Changes in practice performance as a result of the application of what was learned.

Impact on patient health status due to practice behavior changes

Impact on population health status due to changes in practice behavior

*Davis, D, Barnes, BE, & Fox, R (2003). The Continuing Professional Development of Physicians, From Research to Practice. AMA Press. P.251

PI CME (diabetes example below)

Measuring Change – Performance Improvement Model

Additive Measurement Tools

Bar coding allows us to match learner data across multiple assessment stages, reflecting change over time.

Aggregating Data Across Learner Sets

Linking Needs Assessments to Outcomes

• Can outcomes be measured for all activities?

– Yes!

• All CME activities have learning objectives

• Outcomes measurements assess how well learning objectives were met – or not!

• Standardization of evaluation forms betweenproviders would allow for interactivity comparisons and aggregate data evaluation

• Developing an outcomes plan

Outcomes Measurement Methods - Examples

• Single activity – live

– Pre- and post-activity questions demonstrates change from baseline

– Post-activity follow-up shows retention; can be done at varying intervals BUT not ideal

– Case studies – allows for comparison between participants and non-participants

• Multiple activities

– All can be done, but comparisons can be made

Level of Measurement Varies Based on Approach

• Patient and population level change not always attainable directly– Indirect measures using case studies

have been shown to be accurate reflections of practice level changes

• All levels of measurement serve a purpose

Moving to More Sophisticated Approaches• Use of various behavior models to assess practice

variation and develop educational interventions – Stage of change

– Diffusion and Adoption theory

– Precede-Proceed Model

– Social cognitive theory

– Theory of Reasoned Action

– Systems Theory

• Use of framework allows for– Development of outcome measures

– Development of strategic interventions and not just tactics

– Explanation of results

– Assessment of influential factors in decision-making

Section 3 – Technology in CME/CPD

Section 3 – Technology in CME/CPD

Question: What technology are you currently using in CME/CPD?

Section 3 – Technology in CME/CPD

Question: What new or innovative technology has a place in

CME/CPD?

eCME Could And Should Simulate Clinical Practice Environments

Technology Supports Multiple Platforms

Multiple Integrated Online Models

• Used for self-assessment measurements of knowledge and competency

• Used to deliver MicroCME, small “bursts” of education and reminders

• Used to deliver text messages to clinicians and patients alike

Smartphone Apps

• 2,031 HCPs participated in live CME

• 487 patients participated in text messaging, receiving an aggregated total of 44,841 text messages on FM

Smartphones and CME

• Used for self-assessment measurements of knowledge and competency

• Used to delivery MicroCME, small “bursts” of education and reminders

• Used for delivering text messages to clinicians and patients alike

Smartphone Apps

• NOF Guidelines App, a joint development project

Smartphones and CME

• Used for self-assessment measurements of knowledge and competency

• Used to delivery MicroCME, small “bursts” of education and reminders

• Used for delivering text messages to clinicians and patients alike

Smartphone Apps

• MD Self-Assessment App, measuring knowledge and awareness of chronic pain management

Smartphones and CME

Is There A Place For Social Networking in Medicine and CME?

• Physicians and healthcare providers need to communicate

– With each other

– With their patients

– With the public

– With other professionals

• Physicians and healthcare providers represent a “community”

• Needs assessments and professional practice gap analyses consistently identify communications as an area of need

• Most importantly: know your audience!

– Not all physicians will want to use social networking

• Of course not all wanted the Internet of email either

– Know barriers, obstacles, and value definitions

– It may or may not be generational

• Think Prochaska readiness to change1

1http://www.uri.edu/research/cprc/TTM/detailedoverview.htm

Big Questions For Many

• Who is going to use it?

• Who is going to pay for it?

• Is it sustainable?

• Who can do it?

• Who should do it?

• Who will do it?

• Who will monitor it?

The Power Of The Platform

• CME can be deployed in any form (video, slideshow, podcast, etc)

• The community drives learning and awareness– Peer-to-Peer interaction influences

learning and behavior change – • Reinforcement of learning through

community acceptance

– Immediate feedback - quality of CME product, etc

The Power Of The Platform

• Behavior change tracking– Track over time

• Capture how each physician behaves, reports, and discusses the topic as time goes on

– Do they evangelize the technique/process/treatment/etc

– Do they ask about it again?

• vs. traditional post CME surveys (one and done)

What About Social Media and CME/CPD?

When I Think of Social Media…

Use of Facebook in A Needs Assessment

Twitter – Many Uses For CME Providers

Twitter – Follow

• #hcsm, hcsmeu, #acme2011

• @cmeadvocate, @europeancme, @meducate

• @asco, #asco10

Use Of Twitter in MY Needs Assessment

Use of LinkedIn in MY Needs Assessment

Use of LinkedIn in MY Needs Assessment

Know Your Audience – I 60% of Physicians are Interested in Physician Social Networks

65

hello

Manhattan Research – Taking the Pulse v8

Know Your Audience - II

• Q1 2008 telephone and online survey of 1,832 practicing U.S. physicians

• Sermo and Medscape Physician Connect are the two largest physician-only online communities

– Each has about 100,000 users

– Physicians participating in such online communities are more likely to:

• Be primary care physicians

• Be female

• Own a PDA or smartphone

• Go online during or between patient consultations

• Be slightly younger than the average physician

Manhattan Research – Taking the Pulse v8

Know Your Audience - III

• Examples of who is using Twitter?

– Government

• CDC

• FDA

– Specialty societies

• AHA

• NKF

– Healthcare providers

• GICareCenter

• GoSleepSeattle

– Supporters/Pharma

• BI - Novartis

• AZ - JnJ

• Roche - Pfizer

Know Your Audience – IV

The Patient IS Involved

Use of CDC YouTube Site During H1N1 Pandemic

CDC had less than 1,000 Twitter followers in March. They now have over 500,000

70

CDC had <1,000 Twitter followers in March 2009—now they have >1,000,000 for @CDCEmergency

CDC’s H1N1 video has over 2,100,000 views!

"Web-based mapping, search-term surveillance, "microblogging," and online social networks have emerged as alternative forms of rapid dissemination of information."

-New England Journal of Medicine on May 7, 2009

And “Your” Audience Grows – Friending and Retweeting

Found On Twitter – Bias Or JIT Reporting?

Found On Twitter – MDs And Twitter

http://www.annemergmed.com/article/S0196-0644%2809%2900613-1/fulltext

Has Social Networking Existed With Different Names?

• Perhaps!– Physicians Online – 1990s

• Online community through subscriptions• Interactions• Short of networking

– Medscape• Huge community – global• Limited interactions• No networking per se

– Other similar groups• MedPage Today• Epocrates

– Interactions were driven by the “sites” therefore one-way• What about specialty society web sites/offerings?

– Maybe…

Real Social Networking Emerges…

• SERMO– Original mission: Adverse events reporting– Moved quickly to: Multi-use, physician only community– No advertising– Funding model: selling data to pharma, govt, etc.

• Secondary funding model: access to CME providers for audience generation, participation, measurement

– Had partnership with AMA; ended in July 2009• Ozmosis• Syndicom• LinkedIn• Others where communities already exist

– Add in where the community has a need– Useful in CME at many levels

CME and Social NetworkingAt the Moment

• The use of social networking in CME is still young• Activities that have used various aspects of social

networking have only recently been completed or haven’t taken place yet

• The initial inclusion of social networking in CME may not have been best practices

• Those with the data may not be sure of what they have• Those with the data may not be ready to share• The most empiric use of social networking in CME

will be when CME providers incorporate it as an appropriate adjunct at all relevant time points in the lifecycle of CME activities and programs

LinkedIn CME Group – Are You Members?

Section 3 – Technology in CME/CPD

Question: Will technology-based CME/CPD replace some or all of

live CME/CPD?

Online CME – Present and Future

• Currently 6-8% (as of 2008)

• 76% is housed on 16% of sites surveyed

• 70% is $10 or less

• 60% developed by publishing or private medical education providers

• Will make up >50% of all CME in next 8-10 years

Harris JM, et al. JCEHP Winter 2010, 30(1) 3-10

Impact of eCME in Europe – 2010 ECF

http://www.pmlive.com/find_an_article/allarticles/categories/pr_and_

med_ed/2010/november/features/navigating_the_minefield

Where Does eCME Fit in European Mix – 2010?

http://www.pmlive.com/find_an_article/allarticles/categories/pr_and_

med_ed/2010/november/features/navigating_the_minefield

Section 4 – Using a Curricular Approach

in CME/CPD

Section 4 – Using a Curricular Approach

in CME/CPD

Question: Has CME/CPD traditionally followed a true

curricular approach?

Section 4 – Using a Curricular Approach

in CME/CPD

Question: What is needed to transform CME/CPD to a curricular

model?

Section 4 – Using a Curricular Approach

in CME/CPD

Question: Will physicians participate in CME/CPD curricula?

Section 4 – Using a Curricular Approach

in CME/CPD

Question: Does a curricular approach support performance

improvement in CME/CPD?

Multiple Channel Delivery is Key to Effective Reach and Impact

• Education should be provided to learners where learners seek education

• Objectives of activities should be consistent with objectives of channel

Rationale

• Based on linking needs data to outcomes

• Delivery via multiple channels– Based on preferences– Through collaboration where there is a

known interested audience

• Time and budgetary benefits– What is the optimum balance of channel

distribution?

Multiple Formats Serve All Learning StylesOnline | Smartphone | Print | Live & Workshops | MicroCME |

Video

CME Could And Should Use Multiple Platforms

CME Could And Should Provide A Mechanism For Using The Latest Information Presented

Assessment Phase

Quiz and Commentary Activity(Knowledge / Mini Case

Questions with Faculty Commentary)

and self reported frequency of use of practice strategies

Doctor’s Channel VideoData Presentation

RealCME Virtual PatientsData Application to Practice

» Separate 0.50 point CME activity

» Released 60-90 days before conference

» Data can be assessed regionally, by profession and specialty

Conference Post Conference Assessment Phase

Intervention 1 Intervention 2

Email Survey(Knowledge / Mini Case

Questions with Faculty Commentary)

current frequency of

use of practice strategies

»» Released 180 days afterconference

» Data can be assessed regionally, by profession and specialty

Section 4 – Using a Curricular Approach

in CME/CPD

Question: What is the best method for evaluating the overall

impact of curriculum-based CME/CPD?

For Friday

Question: Is there a role for humour in CME/CPD?

Improving Healthcare Worldwide.

ProvaEducation.com