Investigating a blended model of virtual and live simulated patient

32
Investigating and optimizing a blended model of virtual and live patient simulated cases Mary Power, Centre for Teaching Excellence [email protected] Lisa Craig, School of Pharmacy [email protected] with Pharmacy students: Peter Pagacz and Carol Khalil

Transcript of Investigating a blended model of virtual and live simulated patient

Page 1: Investigating a blended model of virtual and live simulated patient

Investigating and optimizing

a blended model of virtual

and live patient simulated

cases

Mary Power, Centre for Teaching Excellence [email protected]

Lisa Craig, School of Pharmacy [email protected]

with Pharmacy students:

Peter Pagacz and Carol Khalil

Page 2: Investigating a blended model of virtual and live simulated patient

Situated in downtown Kitchener,

7 km from rest of UW campus

First graduating class 2011

~ 115 students/year

Page 3: Investigating a blended model of virtual and live simulated patient

Professional Practice Curriculum

Professional Practice (PP) Courses/Labs are the only place

where counseling skills are reinforced.

with standardized patients and peer “patients”.

– 1st year: PP1 and PP2 lecture and PP1 lab

– 2nd year: PP3 lecture and PP2 lab

– 3rd year: PP3 lab (spring** more than one year after PP2 lab**)

– 4th year: PP4 course (spring)

Page 4: Investigating a blended model of virtual and live simulated patient

Co-op • The first co-op term occurs at the end of first year

following (PP1 and 2 lecture and PP1 lab)

• 2nd year starts with the PP3 course and PP2 lab followed by another co-op term

• Students return for an academic term without any “skills” labs, then another co-op term before they return to PP 3 lab.

• It was felt that the co-op terms would be a sufficient way

to ensure that the “skills” were reinforced.

Page 5: Investigating a blended model of virtual and live simulated patient

What happened?

Patient information gathering skills were lost

BUT……

– A key core competency is the ability to safely

and effectively conduct a comprehensive and

accurate patient history.

Page 6: Investigating a blended model of virtual and live simulated patient

Why cases?

Bloom’s Taxonomy

Cone of Learning

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation Teaching Others

Coop Group Work

Demonstrations

Hands On Independ. Study

Lectures

Reading

Visuals

% Retention (6 weeks)

80-90

60-80

15-65

4-8

6-10

12-18

CASE

STUDIES

adapted from http://sciencecases.lib.buffalo.edu/

Page 7: Investigating a blended model of virtual and live simulated patient

What does the literature say?

• simulations such as virtual and standardized patients shown to be beneficial for the development of patient information gathering skills (Chaikoolvatana& Goodyer, 2003) (Schittek Janda et al., 2004) (Yudkowsky, Downing, & Ommert, 2006) (Haist et al., 2008).

• preliminary research also suggests that a blended model, using virtual and standardized patients, may be optimal (Raij et al., 2007).

however little is known about the optimal use of these modalities.

Page 8: Investigating a blended model of virtual and live simulated patient

What do we see as the value of virtual

patients?

• solidify in students’ minds the essential information

gathering questions

• immediate feedback students learn as they go

• students can re-visit reinforce the knowledge

• reusable therefore financially beneficial

Page 9: Investigating a blended model of virtual and live simulated patient

Objectives

Determine if virtual patients will help our students

improve their patient information gathering skills and

what an optimal blend of VP and SP would be

To improve end of term OSCE (Objective Standardized

Clinical Evaluation) scores

Ultimately to best prepare our students for the OSCE

that is a requirement for licensure

Page 10: Investigating a blended model of virtual and live simulated patient

Experimental Design

1. Baseline information gathering skills assessed (in

PP1) pre-OSCE

2. 1st 5 weeks - students interact with cases in various

blends

Page 11: Investigating a blended model of virtual and live simulated patient

Experimental Design

3. 6th week information gathering skills assessed

(post-OSCE) assess change in rubric score

4. 2nd 5 weeks in lab- students interacted with

reverse case blend scenario

5. final OSCE given and assessed compared

with previous years)

6. survey students to ascertain perceived value of

virtual patient cases

Page 12: Investigating a blended model of virtual and live simulated patient

Experimental Design (Phase 2)

7. Trial the optimal blend with students the following

year

8. Determine optimum temporal placement of virtual

and standardized patients

9. Survey students to ascertain perceived value of

virtual patient cases

Page 13: Investigating a blended model of virtual and live simulated patient
Page 14: Investigating a blended model of virtual and live simulated patient
Page 15: Investigating a blended model of virtual and live simulated patient
Page 16: Investigating a blended model of virtual and live simulated patient
Page 17: Investigating a blended model of virtual and live simulated patient
Page 18: Investigating a blended model of virtual and live simulated patient

Did VP# affect student learning?

No significant difference

# VPs Pre-OSCE

(avg) Post-OSCE

(avg) Avg gain

(+sd)

Five (n=17) 48% 52% 4% (+36%

Four (n =16) 54% 50% -4% (+ 22%)

Three (n=12) 48% 60% 12% (+22%)

Two (n=18) 44% 55% 11% (+28%)

One (n=10) 44% 54% 10% (+23%)

Page 19: Investigating a blended model of virtual and live simulated patient

Implementation flaws:

inter-rater reliability

grader training

rubric too complex

poor instructor buy-in

Page 20: Investigating a blended model of virtual and live simulated patient

“Do you agree that virtual patients help in retention/reinforce application/help learning….” (n = 25)

0

2

4

6

8

10

12

14

16

retention reinforce application help learning

strongly agree/agree neutral disagree/strongly disagree

6 respondents from 5 VP group 5 disagree/strongly

disagree for all three categories and one disagreed for all but retention question (agree)

Page 21: Investigating a blended model of virtual and live simulated patient

“I liked them because they helped me to learn the questions that I needed to ask real patients.”

“While I definitely see the value in the use of Virtual Patients in Professional Practice Lab, I think it really important to recognize that they will NEVER replace standardized patients. They are great for "pre-practice", but at the end of the day, the standardized patients is much more similar to what we will be dealing with in real-life.”

“Having 5 virtual patients during the first 5 weeks definitely hindered the progression of my line of questioning. I was basically clicking the questions I thought were right and they were already there for me so I didn't have to do any thinking.”

Do you agree that the virtual patient cases enhance your learning?

Page 22: Investigating a blended model of virtual and live simulated patient

What do you see as the most valuable use of the virtual patients?

0 5 10 15 20 25 30 35

review/study aid

reinforce material learned in class/lab

counseling practice in lab

for testing

not useful at all

Page 23: Investigating a blended model of virtual and live simulated patient

Phase 2: Implementation of “Optimal” Blend

• Decided upon a blend of 3 virtual patients and

3 standardized patients

• Virtual patients were provided on-line ahead of

class for students to work through individually

• In tutorial guided discussion of case

(concentrating on information gathering)

• Standardized patients following week in lab

Page 24: Investigating a blended model of virtual and live simulated patient

“Do you agree that virtual patients help in retention/reinforce application/help learning….” (n = 31)

0

4

8

12

16

20

24

28

retention reinforce applications help learning

strongly agree/agree neutral disagree/strongly disagree

Page 25: Investigating a blended model of virtual and live simulated patient

What do you see as the most valuable use of the virtual patients?

0 5 10 15 20 25

review/study aid

reinforce material

learned in class/lab

counseling practice in

lab

for testing

not useful at all

Page 26: Investigating a blended model of virtual and live simulated patient

“The virtual cases are similar to OSCEs that is great. The patients have a problem and you have to ask the right questions to get to the answer.”

“I liked how some of the patients were more difficult to get information out of. It was more realistic.”

“The fact the the virtual patient interviews provided multiple choice options to choose from in terms of questions for the information gathering made it simpler for us to know what to ask because the options were in front of us.”

“The level of information needed for them was definitely above that of a first year student which made it quite difficult and often I just ended up clicking buttons until it was over”

Do you agree that the virtual patient cases enhance your learning?

Page 27: Investigating a blended model of virtual and live simulated patient

Summary

Virtual patients do have value!

Students like virtual patients but not as replacement to SP

Context very important - if students feel they are losing

out on some face-to-face opportunity satisfaction is

decreased

Cases worked best with guided discussion follow-up

Page 28: Investigating a blended model of virtual and live simulated patient

Going forward:

Create a bank of virtual patient cases

Work on integrating cases with online or F2F guided

discussion more fully into the curriculum

Ongoing monitoring of end-of-term OSCEs

Page 29: Investigating a blended model of virtual and live simulated patient

• Scotiabank Pharmacy

Entrepreneurship Award

• University of Waterloo’s Nicol

Entrepreneurial Award

Competition

• Nigel Stokes E-Launch

Scholarship, Enterprise Co-op @

MBET Centre

Peter Pagacz, Class of 2013

An exciting outcome of the VP story……

Page 30: Investigating a blended model of virtual and live simulated patient
Page 31: Investigating a blended model of virtual and live simulated patient

Acknowledgements

References

• Chaikoolvatana, A., & Goodyer, L. (2003). Evaluation of a multimedia case-history simulation program for pharmacy students. American Journal of Pharmaceutical Education, 67(1)

• Schittek Janda, M., Mattheos, N., Nattestad, A., Wagner, A., Nebel, D., Farbom, C., et al. (2004). Simulation of patient encounters using a virtual patient in periodontology instruction of dental students: Design, usability, and learning effect in history-taking skills. European Journal of Dental Education, 8(3), 111-119.

• Yudkowsky, R., Downing, S. M., & Ommert, D. (2006). Prior experiences associated with residents' scores on a communication and interpersonal skill OSCE. Patient Education & Counseling, 62(3), 368-373.

• Haist, S. A., Lineberry, M. J., Griffith, C. H., Hoellein, A. R., Talente, G. M., & Wilson, J. F. (2008). Sexual history inquiry and HIV counseling: Improving clinical skills and medical knowledge through an interactive workshop utilizing standardized patients. Advances in Health Sciences Education, 13(4), 427-434.

• Raij, A. B., Johnsen, K., Dickerson, R. F., Lok, B. C., Cohen, M. S., Duerson, M., et al. (2007). Comparing interpersonal interactions with a virtual human to those with a real human. IEEE Transactions on Visualization & Computer Graphics, 13(3), 443-457.

This project has been funded by the Learning Initiatives Fund (LIF) of the

University of Waterloo.

Thanks also to Nancy Waite, Associate Director of the School of Pharmacy for

her ongoing support and Lisa McLean, PP Course Demonstrator for logistical

support

Page 32: Investigating a blended model of virtual and live simulated patient

Questions?