Dubrowski draft

The different uses of simulation:

From education to product testing

Adam Dubrowski, PhDLearning and Research Institutes

Wilson CentreDepartment of Paediatrics, UofT

www.wordle.net

Chaos in the BrickyardForscher, BK. Chaos in the Brickyard. Science. 1963 Oct 18; 142(3590): 339.

Applying the Scientific Method and underpinning research methodology to generate evidence that will be used to build simulation based education programs.

General Aim

• Models of Medical Education• Introduction to Simulation• Working Model• Components of Simulation

– Simulators– Educators– Organization

• Future Directions

Outline

Apprenticeship Model

Trainer

Trainee

Pat

ien

t C

are

Shift in responsibility

Ashley, 2000


Trainer

Trainee

Pat

ien

t C

are

Gap

Ashley, 2000


Trainer

Trainee

Pat

ien

t C

are

More people die in a given year as a result of medical errors ( 44,000) than from motor vehicle accidents (43,458), breast cancer (42,297), or AIDS (16,516).

"To Err is Human, Building a Safety Health System" (2000).


Trainer

Trainee

Pat

ien

t C

are

Simulation

Simulation Based Education

Learn fasterRemember longer

Be safer

The replication of a task or an event for the purpose of

education and/or assessment.



Workingmodel

Dubrowski, A., Brydges, R., Satterthwaite, L., Xeroulis, G., Classe, R. American Journal of Surgery. in press.

Busy Environment: Multitasking


Adam Dubrowski

make this busier, to make sure that it looks like a multitasking env.

Technical Performance

Communication

Environment

Clinical Knowledge

Attentional capacity threshold

Additional Learning



Technical Performance

Communication

Environment

Clinical Knowledge

Additional Learning

Attentional capacity threshold


Evidence

Kurahashi et al. Surgery. 2010

Study 1: Does it work?

Can simulation based training help in pre-training?

Practice: Trained SkillsTest

WrittenTest

1 week rest


No practice: Novice

Skills Test: Expert Opinion

05

10152025

Written Test: Score

05

10152025

TrainedNovice Novice

TrainedNovice Novice


simulation performanceimproves


learningenables


simulation


How to use it?


Components

Simulators Educators

Organization

SimulatorsInformation Flow

Framework

Brydges et al., Am J Surg. 2007, Sidhu et al., J Vasc Surg. 2007

Study 2: Realism


High fidelity

Animal model

Low fidelity

Study 2: Realism


Exp

ert

Op

inio

n

05

10152025

Senior

05

10152025

Novice

SyntheticCadaver

Exp

ert

Op

inio

n

Pre-training baseline

SyntheticCadaver

Study 2: Realism

simulation appropriate modelsrequires

Study 3: EducatorsErrors in learning

Xeroulis et al., Surgery. 2007

Study 3: Educators

Xeroulis et al., Surgery. 2007

Exp

ert

Op

inio

n

0

5

10

15

20

25

Post-training Retention

1 monthPre-training baseline

Errors allowedErrors not allowedIndependent learning

Study 3: Educators

simulation educated educatorsrequires

Organization

Moulton et al., Ann Surg. 2007

Optimal schedules

Study 4: Organization


1 2 3 4

1 hour block

1 month

1 2 3 4

1 hour block

1 weekrest

1 month

distributed practice

massed practice

1 2 3 41 2 3 41 2 3 41 2 3 41 2 3 4

T

T

Transfer



distributedmassed

Exp

ert

O

pin

ion

30

20

10

0 Pre-test

Microsurgical drill

Post-test

Rat

Transfer


simulation effective curricularequires

Summary of Results

simulation appropriate modelsrequires

simulation educated educatorsrequiressimulation effective curricularequires

Adam Dubrowski

implications

Implications

Trainer

Trainee

Pat

ien

t C

are

Medical Education = Evidence Based Education

Simulation for Assessment

FrameworkPreferenceFor learningOf learningTesting

Framework

Results

Behavior

Learning

Reaction

Kirkpatrick


Results

Behavior

Learning

Reaction

Kirkpatrick


Results

Behavior

Learning

Reaction

Kirkpatrick

Does

Shows How

Knows How

Knows

Miller


Results

Behavior

Learning

Reaction

Kirkpatrick

Does

Shows How

Knows How

Knows

Miller

Moore


Assessment of what?

Assessment of Performance

Performance

PGY 1-5 Gen Surgery residentsKnot tying: surface (easy) and depth (difficult)Assessed with expert-based (GRS) and computer-based (motion analysis device) assessments

Brydges et al., J Am Coll Surg. 2008


T

T

T

T T T

T T TT

400

200

100

0

5

4

3

2

1

0

T TT

T T

T T T TT

1000

800

600

400

200

0T

T

T

T TSurgical Volume

Expert-basedComputer-based

1 2 3 4 5

DepthSurface



simulation valid assessmentsrequires



simulation realistic contextprovides



Porte et al., AJS. 2007

Assessment for Learning

Computer

Practice

PreTest

RetentionTest

1 week restComputer + Crit

Expert

PostTest



T T

T

T

T TT

T T

Pre Post Retention

Exp

ert

O

pin

ion

20

10

0

Computer

Computer + Crit

Expert




assessments for learning: choices


Assessment of Learning

Learning

Building Skills, Changing Practice: Simulator Training for Hand Hygiene Protocols

Canadian Institutes of Health ResearchPartnerships for Health System Improvement

A. McGeer, MA. Beduz, A. Dubrowski


Purpose

• Current models of knowledge delivery about proper hand hygiene rely on didactic session

• Transfer to practice is low


PurposeAssessment of Learning

Results

Behavior

Learning

Reaction

Does

Shows How

Knows How

Knows

Recruitment

Clinical monitoring (audits)

Clinical monitoring (audits)

Δ in Behavior

Δ in PerformanceReaction

Education(Sim/

traditional)


Sim

Sim

• Simulation Design Scale• Educational Practices Questionnaire

PurposeAssessment of LearningFre

qu

en

cy

100

50

0

Disagree StronglyAgree

90

66

Pre-test Post-testHand Hygiene Compliance(average of all opportunities; all 4 moments)

62% (n=558

opportunities)

64% (n=550

opportunities)

Average Adequacy(Excellent/ Satisfactory/ Unsatisfactory)

Satisfactory (n=252

observations)

Satisfactory (n=282

observations)

Average Duration

13 seconds (n=254

observations)

13 seconds(n=252

observations)

# Extra Washes 140 90


Bef-Pat/Env Bef-Asp Aft-Bfl Aft-Pat/Env0

10

20

30

40

50

60

70

80

90

100 98

24

51

9292

4448

93

PreTest (n=558)Post Test (n=550)

Moment 1 Moment 2 Moment 3 Moment 4


*

• No transfer of knowledge

• No changes in performance, no learning

• Ineffective education and not transfer to practice


Assessment for Testing

Product and Process testing

Patient handoffs


Patient handoffs Checklist


Is it an effective tool?Does it disturb the flow?What is the frequency of use?What is the most optimal format?



simulation with process testinghelps


Summary

Summary

Best available evidence

Ongoing assessment

Educator’s experience

• Simulators• Educators• Schedule

Summary


Ongoing assessment



• Performance• For learning• Of learning• Other uses

Summary


Ongoing assessment



• Performance• For learning• Of learning• Other uses

Simulation may not be the

answer

Dubrowski draft

Education

Transcript of Dubrowski draft