Learning Objectives and Learning Objectives and Other Useful InformationOther Useful Information

Theresa Barrett, MS, CMPTheresa Barrett, MS, CMPDeputy EVP

New Jersey Academy of Family Physicians

22

What needs to be in a What needs to be in a needs assessment?needs assessment?

““My Education Committee said so” is not My Education Committee said so” is not considered a component of a needs considered a component of a needs assessmentassessment

33

What needs to be in a What needs to be in a needs assessment?needs assessment?

Let’s start with a definitionLet’s start with a definition A needs assessment is a process of identifying and A needs assessment is a process of identifying and

analyzing data that reflects the need for a particular analyzing data that reflects the need for a particular CME activityCME activity

The data could result from a survey of the potential The data could result from a survey of the potential learners, evaluations from previous CME activities, learners, evaluations from previous CME activities, needed health outcomes, identification of needed new needed health outcomes, identification of needed new skills, literature searchesskills, literature searches

Needs assessment data drives the educational Needs assessment data drives the educational objectivesobjectives

44

Use multiple sourcesUse multiple sources

Using multiple sources for your needs Using multiple sources for your needs assessment gives creditability to your research assessment gives creditability to your research and provides a better understanding of needsand provides a better understanding of needs

55

Where to find the dataWhere to find the data

Survey results Survey results AAFP annual educational needs assessmentAAFP annual educational needs assessment

Available online (see handout)Available online (see handout)

66

Where to find the dataWhere to find the data

Evaluations from previous CME activitiesEvaluations from previous CME activitiesDo more than just ask what your members want Do more than just ask what your members want Craft the question to gather data you can useCraft the question to gather data you can useExample 1: Example 1:

Q: I would like additional education onQ: I would like additional education on A: [fill in the blank]A: [fill in the blank]

Example 2Example 2 Current research shows aggressive treatment of lipids Current research shows aggressive treatment of lipids

leads to reduced CV risk. leads to reduced CV risk. __It is important to me to learn more about th6is__It is important to me to learn more about th6is __I would like more education on this__I would like more education on this __I know all I need to know on this topic__I know all I need to know on this topic

77

Where to find the dataWhere to find the data

Identified health outcomes Identified health outcomes 88thth Scope of Work, CDC, DHSS Scope of Work, CDC, DHSS

Identified new skills Identified new skills New guidelines to be followedNew guidelines to be followed

Literature review Literature review PubMed PubMed

88

Where to find the dataWhere to find the data

Quality Improvement (QI) dataQuality Improvement (QI) dataAHRQ, P4P, HEDISAHRQ, P4P, HEDIS

Federal/State government mandateFederal/State government mandateMeningococcal vaccine mandated for college Meningococcal vaccine mandated for college

admission in NJ. Physicians need more admission in NJ. Physicians need more information to council patients/parentsinformation to council patients/parents

OthersOthersCDCCDCPatient support websitesPatient support websites

American Heart AssociationAmerican Heart Association American Cancer SocietyAmerican Cancer Society

99

Is this a needs assessment?Is this a needs assessment?

The University of California’s The University of California’s Review and Review and Update of Pain and Palliative MedicineUpdate of Pain and Palliative Medicine is is designed to help front-line professionals designed to help front-line professionals better care for their patients who are suffering better care for their patients who are suffering from pain. Areas covered include an overview from pain. Areas covered include an overview of pain, pain syndromes, cancer pain & of pain, pain syndromes, cancer pain & palliative care, pediatric pain, methods of palliative care, pediatric pain, methods of treatment, addiction, and legal issues in pain treatment, addiction, and legal issues in pain medicine.medicine.

1010

Well, Not ReallyWell, Not Really

This is a good This is a good overviewoverview, but it does not confirm the , but it does not confirm the needneed for such an activity. for such an activity.

What could help?What could help? California has mandated that all physicians receive California has mandated that all physicians receive

CME in pain and palliative care to renew licensureCME in pain and palliative care to renew licensure Data about social disparities in treatment of pain Data about social disparities in treatment of pain New research evidence about undertreatment of painNew research evidence about undertreatment of pain Other suggestions?Other suggestions?

1111

Better Better

Research has shown that social disparities Research has shown that social disparities exist in the treatment of pain exist in the treatment of pain 1,2,3. 1,2,3. In addition, In addition, the state of California has mandated that the state of California has mandated that all physicians receive 10 credits in all physicians receive 10 credits in continuing medication education in pain continuing medication education in pain and palliative care to renew licensure. and palliative care to renew licensure. Therefore The University of California will Therefore The University of California will offer a CME activity entitled offer a CME activity entitled Review and Review and Update of Pain and Palliative MedicineUpdate of Pain and Palliative Medicine is is address these needs.address these needs.

1212

Learning ObjectivesLearning Objectives

The needs assessment drives the learning The needs assessment drives the learning objectivesobjectives

Learning Objectives should clearly Learning Objectives should clearly describe what the learner will be able to describe what the learner will be able to know or do after participating in the CME know or do after participating in the CME activity. activity.

Good objectives are SMARTGood objectives are SMART

1313

Learning ObjectivesLearning Objectives

SSpecificpecificMMeasurableeasurableAAttainablettainableRResults-Focusedesults-FocusedTTime-Focused ime-Focused

1414

Learning ObjectivesLearning Objectives

S = SpecificS = SpecificConcrete Concrete Use Action WordsUse Action Words

Original Objective:Original Objective:The University will facilitate continuing education The University will facilitate continuing education for pain management.for pain management.

Specific Objective:Specific Objective:The Academy will create a web-based, interactive The Academy will create a web-based, interactive site… site…

1515

Learning ObjectivesLearning Objectives

M = MeasurableM = MeasurableNumeric or descriptive Numeric or descriptive Quantity, quality, cost Quantity, quality, cost

1616

Learning ObjectivesLearning Objectives

A = ActionableA = ActionableFeasible Feasible Appropriately limited in scope Appropriately limited in scope Within the learner’s control and influence Within the learner’s control and influence

1717

Learning ObjectivesLearning Objectives

R = RealisticR = RealisticAble to be accomplishedAble to be accomplished

1818

Learning ObjectivesLearning Objectives

T = TimelyT = TimelySays when the objective will be accomplishedSays when the objective will be accomplished

1919

Action VerbsAction Verbs

CiteCite DemonstrateDemonstrate CountCount AnalyzeAnalyze DescribeDescribe FormulateFormulate

AssessAssess DistinguishDistinguish DiagnoseDiagnose DiagramDiagram MeasureMeasure SummarizeSummarize

See additional suggestions in handoutSee additional suggestions in handout

2020

Words to avoidWords to avoid

Understand Know Realize Be aware of Perceive

2121

Are these good Are these good learning objectives?learning objectives?

““Global Objectives” for a grand rounds Global Objectives” for a grand rounds series. series. Participants can expect to learnParticipants can expect to learn

preventive measures and screeningspreventive measures and screeningsevidence-based treatment options for specific evidence-based treatment options for specific

diseasesdiseasescurrent issues in medical fieldcurrent issues in medical field

2222

Suggested ImprovementSuggested Improvement

At the conclusion of each grand rounds session, At the conclusion of each grand rounds session, learners should be able to:learners should be able to:Describe preventive measures and Describe preventive measures and

screenings related to the session topicscreenings related to the session topicUtilize evidence-based treatment options to Utilize evidence-based treatment options to

treat the session topic disease or condition, treat the session topic disease or condition, and/orand/or

Discuss current issues related to the session Discuss current issues related to the session topictopic

2323

Are these good Are these good learning objectives?learning objectives?

Learning objectives for an annual primary Learning objectives for an annual primary care update activitycare update activity Participants can expect toParticipants can expect to

Learn about current topics in primary careLearn about current topics in primary careFocus is on practical information of use to the Focus is on practical information of use to the

active MDactive MD

2424

Suggested ImprovementSuggested ImprovementAt the conclusion of this activity, learners At the conclusion of this activity, learners

should be able to:should be able to:Discuss new research in the identification and Discuss new research in the identification and

treatment of fibromyalgia treatment of fibromyalgia Describe three methods of treatment of STDs Describe three methods of treatment of STDs

in adolescentsin adolescentsReview the current recommendations for the Review the current recommendations for the

use of antibiotics in adultsuse of antibiotics in adultsCompare and contrast the differences in Compare and contrast the differences in

treatment for rosacea and urticariatreatment for rosacea and urticaria

2525

RememberRemember

Learning objectives should inform the learner Learning objectives should inform the learner what he can expect to get out of attending the what he can expect to get out of attending the activity. activity.

Used correctly, strong learning objectives can be Used correctly, strong learning objectives can be an excellent “selling point” for your activityan excellent “selling point” for your activity

2626

Educational EffectivenessEducational EffectivenessAdult Learning Principles

LESS EFFECTIVE MORE EFFECTIVE

Faculty-centered Learner-centered

Passive participation Active engagement

Knowledge-focused Application-focused

Generalized content Personalized content

One media method Multiple media methods

One intervention Multiple interventions

Sources: AHRQ, Grimshaw et al, and many others

2727

Educational (PI) EffectivenessEducational (PI) EffectivenessVarious Methodologies

ED / PI METHOD IS IT EFFECTIVE ?Outreach Yes IF … done with audit + feedbackAudit + fdbk Yes IF … low baseline + high intensityReminders Yes IF … right info/right time is neededKOLs Varies by leader + topic

Clin guides Yes IF … physician buy-in + org supportConferences No UNLESS … interactive, need-drivenPrint info Usually NOT effective$ incentives Can be IF … high enough, credible

Sources: Baughner, Bloom, Davis, Freemantle, Jamtvest, Overstreet, Oxman

2828

Evaluation LevelsEvaluation LevelsEducational Outcome Measurement

Participation (attendance)

Population Health

Satisfaction (learner reactions)

Learning(knowledge, skill, attitude, aptitude)

Performance(change in behavior)

Patient Health

Adapted from: Moore DE, Kirkpatrick, Barr et al

2929

Evaluation MethodsEvaluation MethodsMeasuring Educational Outcomes

Outcomes Measurement MethodsParticipation - Attendance rostersSatisfaction - Activity evaluation ratingsLearning - Pre-post tests (knowledge, skill, confidence)

- Self, peer, expert assessments

Performance - self-reporting

- chart pulls, practice audits, HEDIS data

Patient Health - Case-specific outcomes (e.g., LOS, QOL, re-admissions, complications, immunizations)

Population Health

- Disease incidence, morbidity/mortality rates

3030

Choose Evaluation MethodsEvaluation MethodsBased on Learning ObjectivesLearning Objectives

ImmediateImmediate Learning: Learning: Compare pre vs. post interventionCompare pre vs. post intervention Compare case vs. control performanceCompare case vs. control performance

PersistentPersistent Learning: Learning: Use multiple approaches & interventionsUse multiple approaches & interventions

AppliedApplied Learning: Learning: Measure impact on behavior / practiceMeasure impact on behavior / practice

Source: http://ccoe.umdnj.edu/general/outcomes.htm 1/15/07

3131

Satisfaction SurveysSatisfaction Surveys(NOT RECOMMENDED)

Evaluation of faculty / activity (Evaluation of faculty / activity (NOTNOT learning) learning)ContentContentDeliveryDeliveryPractice RelevancePractice RelevanceHandouts and AVHandouts and AV

““To what extent were you pleased with… ?”To what extent were you pleased with… ?” ““How can we improve in the future ?”How can we improve in the future ?”““Other speakers / topics to suggest ?”Other speakers / topics to suggest ?”

3232

Self-Assessed Learning SurveysSelf-Assessed Learning Surveys(Better)

To what extent was ___ learning objective met?To what extent was ___ learning objective met?

To what extent are you more To what extent are you more [aware of, prepared to, confident about, proficient at][aware of, prepared to, confident about, proficient at] ___? ___?

To what extent are you To what extent are you [planning to, prepared to, [planning to, prepared to, committed to]committed to] apply this learning in your practice? apply this learning in your practice?

When / how will you measure the impact on your … When / how will you measure the impact on your … [practice or patients][practice or patients]

3333

Evaluation Evaluation shouldshould be … be …

A basis for continuous improvement:A basis for continuous improvement:UsefulUseful educational outcome data educational outcome dataNewNew needs assessment data needs assessment data InsightsInsights for planning future educational activities for planning future educational activitiesVerificationVerification that content is relevant and unbiased that content is relevant and unbiased

Not just a “necessary requirement”Not just a “necessary requirement”