Rural Palliative Care (PC) Education: Results of a

Hybrid Course with Face-to-Face and Online

Learning.

Dr. Jose Pereira•Alberta Cancer Foundation

Professor of Palliative Medicine

University of Calgary

Goals of the Project

• Instil residents with competencies required to care for terminally ill patients.

• Develop evaluation tools– knowledge, attitudes and skills.

• Explore residents’ responses to the inclusion in the curriculum of:– Spirituality, hope, suffering, self-

awareness, use of narrative, humanities to engage affective domain of learning

Course Design

• Hybrid model: F2F & online

OSCEs (x4) & F2F workshops

10 weeks

Online

1½ days

1½ days F2F workshops & OSCEs (x4):

Course introTechnology intro.CommunicationPain/SymptomsInteractive, schemes & case-based

CommunicationDecision-making

EthicsPain/Symptoms

Movies & arts

Course Design

10 weeks

Online

Assignments

(E-mail)

Case Discussions

Thematic Discussions

Ask the Expert

4 modulesEach

2-weeks long

Small group asynchronous discussions

Evaluation Framework• Knowledge

– Pre vs Post-course knowledge test – 20-items, multiple choice questionnaire– Based on blueprint; face validity– Varying Bloom’s hierarchies

• Attitudes– Surveys– Self-perceived changes in clinical comfort levels

• 22 items, 5-point Likert-like scale (1=not at all comfortable, 5=very comfortable)

– Inclusion of topics in learning– Focus Groups

• Skills– 4 OSCEs

• Course participation• Course itself

– Surveys & Focus Groups

Results: Knowledge

• N=15

• Internal Reliability: Cronbach’s : 0.5– (Need to increase # of items to 30 to increase

reliability to 0.67)

• Significant improvement in knowledge– Repeated measures test: F=19.8, p=0.001– Cohen’s effect size: 0.77– Pre-course mean (SD): 12 (2.6)– Post-course mean (SD): 16 (1.9)

Self-perceived clinical comfort levels: Pre vs Post course

• N=15 • Significant improvement in comfort

levels– Repeated measures test: F=75.3,

p<0.001

– Cohen’s effect size: 0.92– Pre-course mean (SD): 59.7 (10.9)– Post-course mean (SD): 82.8 (4.7)

Self-perceived comfort levels

• Pre versus Post Course– Little change in communication– Large change in pain & symptom

managementBut

• At post course when asked “compared to when you first started…”– Large change in communication as well– Role of OSCEs for self-assessment

Focus Groups Results

• Ambivalence to including psychosocial care in case studies.– “..talk about one topic at a time; not

mix; separate the psychosocial from the clinical”

– “Would have liked to see more clinical stuff” [online]

– “I don’t agree; the patient is a whole person, you cannot separate”

Focus Groups Results

• Ambivalence to spirituality in care “Physicians should address spirituality when

treating palliative patients…one cannot separate the physical and the spirit.”..but no-one has taught us how to do this

“For now, we want to learn more about fundamentals of medicine rather than spirituality”

How should we introduce spirituality?

• Perhaps in disguise

Possible Roles of OSCEs

Education tool

Needs Assessment

Formative evaluation

Summative evaluation

Developed from real cases3 domains in each OSCE: physical issues, psychosocial issues & communication. (clinical decision-making & communication)Reflect major competencies

58 y/o university professor with breast cancer. Presents with cancer pain.

• Cancer pain management. Address fears of opioids, explore illness experience.

Young 32 y/o with advanced gastric cancer, nausea & vomiting from upper GI obstruction.Young children.

•Manage psychological distress, being in presence of suffering, managing nausea & vomiting.

60 Y/o man with severe shortness of breath from advanced ALS. Accompanied by wife.

• Explore fears, advanced planning & discuss code status, home care needs, manage dyspnea.

Office visit by home care nurse• Interdisciplinary collaboration, manage delirium, inability to swallow & hypercalcemia in home setting

4 OSCEs in this Course

Steps in developing OSCEs1. Identify competencies & blueprint2. Develop OSCEs (as a team)3. Review OSCEs with content experts & potential

learners (sample from target group of learners.)4. Prepare score sheets

a. Checklist & Global Rating Scale.5. Train actors & actresses 6. Prepare logistics for implementation.7. Test OSCEs with actors/actresses8. Do OSCEs (videotape)9. Rehearse scoring with scorers10. Preliminary reliability testing11. Scoring12. Modifying OSCEs.

Checklist vs Global Rating Scale?

• Opted for checklist & global rating• Literature

– Global rating scales scored by experts showed higher inter-station reliability, better construct & concurrent validity than did checklists.

– The use of checklists prior to using a global ratings scale did not improve the reliability or validity of the global rating.

Regehr G, et al. Acad Med 1998;73:993-997

Scale Design

1. Separate score sheet for each OSCE2. Scale consists of two subscales:

– Performance of Skill– Degree to which skill performed– Items rated on a 3-point scale

3. Criterion-based scoring [Doig et al; Thompson et al]

– Omitted, performed but not competently, performed competently

Results• Inter-rater Reliability based on 4 raters• Cronbach’s Alpha

1. Performance of Skill: .77 - .882. Degree of Skill: .72 - .883. Overall Scale: .87 to .92

• Further inter-rater and intra-rater reliability and generalizability being assessed.

What residents thought of OSCEs

• Very useful learning tools. • Helped them identify their learning

needs and provide them with practice.

• Would recommend it to other residents

Overall Course Evaluation

• Would recommend it to future residents

• Want practical approaches, not theoretical discussions

• Want more mentoring• Some ambivalence about:

– Online learning component – Thematic discussions

• Psychospiritual issues

Strengths & limitations

• Limitations– Small numbers limit generalisability

Conclusions

• There is a culture that does not value integrated care- need to address this in the undergraduate curriculum

• Evaluation methods require careful thought and expertise