CLINICAL SKILLS MANAGED EDUCATIONAL NETWORK

RESEARCH AND DEVELOPMENT GRANTS 2012/13

RESEARCH REPORT

Title

Medical Students Driven to Distraction.

Improving patient safety teaching through the use of

a simulated ward round experience: a prospective control study

Author

Ian Thomas

Highland Medical Education Centre, University of Aberdeen,

Centre for Health Sciences, Old Perth Road, Inverness.

Contributors Ian Thomas

Clinical Teaching Fellow

Laura Nicol

Simulation Fellow

Luke Regan

Clinical Teaching Fellow

Drieka Maliepaard

Clinical Skills Facilitator

Lindsay Clark

Clinical Skills Facilitator

Kenneth Walker

Consultant surgeon

John Duncan

Consultant surgeon

The author would like to thank the Clinical Skills Managed Educational Network for its

generous support – without which this study would not have been completed.

1

MEDICAL STUDENTS DRIVEN TO DISTRACTION

IMPROVING PATIENT SAFETY TEACHING THROUGH THE USE OF

A SIMULATED WARD ROUND EXPERIENCE: A PROSPECTIVE CONTROL STUDY

ABSTRACT

INTRODUCTION

Medical error is common, with poor situational awareness an important aetiological dynamic.

This is a key consideration for junior doctors, for whom distraction and interruption are

endemic in clinical work.

RESEARCH QUESTIONS

The objectives of this study are:

• To explore in a simulated ward experience, if final year medical undergraduates are

able to manage realistic ward-based distractions in order to mitigate medical error.

If this is not the case, then:

• Can a simulated ward experience with feedback be used to improve distractor

management to reduce medical-error making behaviour?

METHODS

A simulated ward round experience for undergraduates was designed with a focus on

distraction and medical error. Conducted as a prospective control trial, 14 students were

enrolled to an intervention group. Students undertook a baseline ward round and received

feedback focused on distractor management and medical error. Four weeks later, students

2

participated in a post-intervention ward round. An independent group of 14 students were

recruited to a control group – who underwent the same experience, but with no targeted

feedback between simulations. Error rates and distractor management was assessed during

simulations.

RESULTS

There was significant positive correlation between medical error-making and the

mismanagement of distractions. At baseline total medical error rates were high – with 72 and

76 documented mistakes in the intervention and control groups respectively.

All forms of simulation training reduced medical error post-intervention: with medical error-

making falling by 76% in the intervention group (p-value <0.0001) and 42% in the control

group (p-value <0.0001).

DISCUSSION & CONCLUSION

Medical students are not equipped to manage ward-based distractions in order to mitigate

medical error. These skills can be taught through simulation. This is the first study of its

kind to demonstrate an undergraduate teaching intervention with improvement in patient

safety behaviour. Curricular integration of the simulated ward round experience is

recommended.

3

INTRODUCTION

Despite best intentions, many patients worldwide suffer avoidable iatrogenic harm. In the

United States 98,000 deaths per year are directly attributable to medical error. 1 Worldwide,

this equates to 6 daily aircraft crashes without survivors. 2 Medical error has persisted

throughout the decades, 3, 4, 5 despite significant technological advances – and arguably non-

technical skills 6 are now the rate-limiting step. A breakdown in these skills is responsible for

56-82% of healthcare-related errors. 7&8 Healthcare professionals must be grounded in non-

technical skills and in particular situational awareness – as dealing with interruption and

multi-tasking are an integral part of the job. 9, 10, 11, 12 Clinician-stress has been shown to

heighten clinical error 13, 14, 15 and indeed distraction plays a central role. 16, 17 It is junior

trainees who report the highest levels of stress in clinical practice. 18 Unfortunately, stress

conversion into medical error is also highest amongst trainee cohorts, 19, 20 a group that feels

ill-prepared by medical school curriculums for managing heavy workloads and prioritising. 21

Medical educators should seek innovative learning experiences to teach non-technical and

patient safety skills to undergraduates. This is in line with the World Health Organisation’s 22

Patient Safety Curriculum and advice from the General Medical Council 23 endorsing the use

of simulated training environments. Simulation has contributed to our understanding of the

detrimental clinical impact of interruption on adherence to protocols, 24 performance in

emergency scenarios, 25, 26 clinical reasoning 27 and accurate calculating of drug doses. 28 As

new graduates will initially work within a ward setting – the environment most prone to

clinical error 29 it is logical to consider a simulated ward experience as a potential teaching

modality.

Nikendi 30 argues that:

4

‘‘Ward round training which eases the transition from observing ward rounds to conducting

them on one’s own is urgently required.’’

There has since been a trend to greater utilisation of simulated ward experiences 31, 32 with

newer generations 33, 34 concentrating on the impact of ward-based distraction. However,

these have merely documented participant acceptability and quantified baseline medical error

rates.

Although the body of evidence is growing, it stops short of demonstrable behavioural change.

Research which shows that simulated ward rounds can achieve this is required.

The objectives of this study are:

• To explore in a simulated ward experience, if final year medical undergraduates are

able to manage realistic ward-based distractions in order to mitigate medical error.

If this is not the case, then:

• Can a simulated ward experience with feedback be used to improve distractor

management to reduce medical-error making behaviour?

5

PEDAGOGICAL DEVELOPMENT OF THE SIMULATED WARD EXPERIENCE

A simulated ward round experience was created at the University of Aberdeen for final year

medical students with a focus on distraction and medical error.

Medical students played the part of a Foundation doctor leading a 30-minute ward round,

consisting of three patients, as shown in Figure 1.

Figure 1 – Role allocation during simulation

Patient with pneumonia:

played by a volunteer

patient.

Foundation doctor: played

by a final year medical

student. Leads the ward

round, makes diagnoses and

appropriate management

plans.

Staff nurse: played by a clinical skills

facilitator. Gives an appropriate

handover of patients to the doctor

and assists with execution of

management plans.

Patient notes

Video-recording equipment

6

Three patient scenarios were designed:

• An elderly lady with pneumonia

• A post-operative male patient with chest pain

• An elderly diabetic male with confusion

At each bed space the doctor was expected to complete a number of clinical duties, with

associated potential medical errors as shown in Figure 2.

As shown in Figures 3 and 4, a set of six realistic, time-critical distractions were deployed.

During simulation the student’s performance was assessed with medical errors and distractor

management recorded on a standardised checklist.

7

Figure 2 – Expected task completion and associated errors during the ward round

Expected task completion

Potential associated medical errors

At start of the simulation

Correctly prioritizes patients (chest pain,

followed by pneumonia then cognitive

impairment)

Does not correctly prioritize patients

Bed 1 – Clinical Problem: Pneumonia

Correctly diagnoses pneumonia

Fails to reach or gives incorrect diagnosis.

Staff nurse has to prompt with diagnosis

Utilizes blood results to calculate patient’s

CURB-65 score

Does not recognize that blood results in notes

do not correspond to correct patient

Correctly calculates CURB-65 score as a

marker of disease severity

Incorrectly calculates CURB-65 score and

nurse must provide appropriate score

Prescribes appropriate antibiotic therapy for

patient based on ward protocol

Fails to recognize patient is allergic to first-

line therapy

Correctly checks antibiotic vial with nurse

ahead of medication administration

Authorizes administration of date-expired

medication vial

Bed 2 – Clinical Problem: Post-operative chest pain

Correctly diagnoses non-ST elevation

myocardial infarction (NSTEMI)

Fails to reach or gives incorrect diagnosis.

Staff nurse has to prompt with diagnosis

Prescribes appropriate therapy for NSTEMI

based on ward protocol

Fails to appreciate patient is post-operative

and anti-coagulation is contraindicated

Nurse asks doctor to prescribe Paracetemol

for patient on regular Co-codamol.

Fails to recognize patient receiving Co-

codamol and Paracetemol contraindicated

Bed 3 – Clinical Problem: Diabetic with cognitive impairment

Amends Insulin dose as per recommendation

in notes from diabetic specialist nurse

Misreads handwritten entry in notes as 25

units: and not 2.5 units: resulting in overdose.

Deals with upset relative who comes into the

ward wanting to speak to the doctor

Does not check relative’s identity and

discloses information on incorrect patient

8

Figure 3 – Ward round facilitation with distraction deployment

Facilitators and assessors sat behind the one-way mirror overlooking the simulated

ward. Time-critical distractions were orchestrated from the observation suite with

communication established through the use of ear pieces and microphones.

One-way mirror Ear piece

Microphone

Timed-distractions

checklist

IT equipment to

facilitate

simulation

Doctor answering his page during simulation

Domestic hoovering at patient bedside as

doctor prescribes

Ward radio switched on at start of simulation

exercise

9

Figure 4 – List of standardised time-critical distractions

Distraction Time-critical deployment Potential associated medical

error

Ward radio switched on

(at 70 decibels)

At the start of the ward

round

Doctor incorrectly prioritizes

order of patients to be seen

Domestic hoovers at the

patient bed space

When the doctor is

reviewing the antibiotic

protocol for chest infection

Doctor prescribes protocol-

driven antibiotic to which

patient is allergic

Doctor’s pager set off

When doctor is reviewing

the chest pain protocol

Doctor prescribes protocol-

driven anti-coagulation and

fails to appreciate patient is

immediately post-operative

Nurse asks doctor to

prescribe Paracetemol for

separate unrelated patient

As doctor is prescribing

protocol-driven chest pain

medication on drug kardex

Prescribes anticoagulation +/-

makes any other related

prescription error

Telephone call into the ward

As doctor is reviewing the

diabetic specialist nurses’

written request in the

medical notes to increase

dose of Insulin to 2.5 units

Misreads handwriting and

prescribes an overdose of 25

units of insulin instead of

desired 2.5 units

Dealing with upset relative

who enters ward and wishes

to speak with the doctor

As student is prescribing

change of Insulin dose on

drug kardex

Fails to check identity of

relative and discloses

confidential information

pertaining to a different

patient

10

METHODS

A prospective control study was conducted, as shown in Figure 5. 14 students were recruited

to an intervention group by simple randomisation and undertook a baseline ward round.

Post-simulation students received immediate individualized feedback, targeted around

distractor management and its relationship to medical error using SET-GO criteria. 35

Following a lag time of 4 weeks students completed a post-intervention ward round of

comparable rigour, achieved by modification of the baseline ward round’s surface

characteristics. Separately, 14 students were similarly recruited to a control group who

undertook the same experience, but did not receive targeted feedback on distraction and error.

Medical error rates and distractor management was assessed at all simulations.

11

Figure 5 – Study design flow diagram

September

2013

Eligibility: 26 final year medical

students at University of Aberdeen

Eligibility: 25 final year medical

students at University of Aberdeen Volunteers: 20 students Volunteers: 22 students

Recruitment: 14 students Recruitment: 14 students

Baseline

Ward round

Baseline

Ward round

Targeted

feedback No targeted

feedback

Post-intervention

ward round

Post-intervention

ward round

Dropout rate:

Intervention group N=0

Control group N=0

Data analysis: 14 students Data analysis: 14 students

Exclusion rate:

Intervention group N=0

Control group N=0

October

2013

November

2013

December

2013

12

RESULTS

Study demographics

The intervention and control groups were comparable across baseline demographics as shown

in Figure 6:

Figure 6: Baseline demographics between intervention and control groups

Demographic

Intervention Control P-value

Participants 14 14 1.00

Males 5 5 1.00

Females 9 9

Average age 23.5

Standard deviation 2.79

23.71

Standard deviation 2.70

0.8382

Mean number of errors per

student at baseline

5.14

Standard deviation 1.03

5.43

Standard deviation 1.09

0.4816

Mean number of distractions

mismanaged per student at

baseline

2.07

Standard deviation 1.00

2.71

Standard deviation 1.33

0.1591

The relationship between medical error and distractor management

Figure 7 shows the positive correlation that exists between medical error and the

mismanagement of distractions (Spearman’s co-efficient of 0.663).

13

Figure 7: Correlation between medical error and distractor management in the

intervention group

Correlations

Errors made in

Intervention Group

Distractions

mishandled in

Intervention group

Spearman's Rank

Co-efficient

Errors made in

Intervention group

Correlation Coefficient 1.000 .663**

Sig. (2-tailed) . .000

N 28 28

Distractions mishandled in

Intervention group

Correlation Coefficient .663** 1.000

Sig. (2-tailed) .000 .

N 28 28

**. Correlation is significant at the 0.01 level (2-tailed).

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 1 2 3 4 5 6 7 8

Nu

mb

er

of

dis

trac

tio

ns

mis

han

dle

d

Number of medical errors committed

The correlation between medical error-making and distractor management: intervention

group

14

Baseline medical errors and distractor management

In the intervention group a total of 72 medical errors were committed at baseline,

representing a mean of 5.14 errors per student. In the control group baseline medical errors

were similar – with 76 medical errors witnessed (mean 5.43 medical errors per student).

The pattern of error between intervention and control groups mirrored one another closely as

shown in Figure 8.

At baseline distractions were poorly managed in both groups. In the intervention group a

total of 29 distractions were mishandled compared to 38 in the control group. The pattern of

distractor management between groups at baseline is shown in Figure 9.

15

Figure 8 – Overview of medical errors made at baseline

0

2

4

6

8

10

12

14

4

0

14

1

1012

3

13

5

2

6

2

8

0

14

0

10

13

7

14

6

3

1

0

Nu

mb

er

of

err

os

Overview of medical errors made at baseline

Intervention Group Control Group

16

Figure 9 – Management of distractions at baseline

Intervention Group

Control Group

0

2

4

6

8

10

12

14

Radio Hoover Pager Drug chart taskdistraction

Dealing withrelative

Telephone call

98

22

5

3

9

14

66

2

1

Nu

mb

er

of

dis

trac

tio

ns

mis

man

age

d

Management of distractions at baseline

Intervention Group Control Group

17

Change in patient safety behaviours between simulations

At post-intervention medical error making fell in both groups – showing that any form of

simulation benefits patient safety behaviours. The reduction in error-rates between baseline

and post-intervention simulations is shown in Figure 10. In the intervention arm medical

errors post-intervention fell by 76.39% (p-value <0.0001), compared to 42.11% (p-value

<0.0001) in the control arm. Although medical error rates fell in both groups, intervention

conferred an additional 34.28% improvement over control (p-value 0.0016).

Figure 11 gives an overview of the errors that that persisted at the post-intervention

simulation. Figure 12 outlines the change in distractor management between simulations. In

the intervention group distractor management improved by 86.21% (p-value <0.0001):

whereas in the control group there was a 2.63% improvement (p-value 0.7929).

18

Figure 10 – Change in error making rates between baseline and post-intervention simulations

0

10

20

30

40

50

60

70

80

Total errors in Interventiongroup

Total errors in Controlgroup

Life threatening errors inIntervention group

Life threatening errors inControl group

7276

2527

17

44

6

18

Nu

mb

er o

f er

rors

co

mm

itte

d

Reduction in medical error rates between baseline and post-intervention simulations

Baseline Post-intervention

19

Figure 11 – Overview of medical errors made at post-intervention

0

2

4

6

8

10

12

14

0 0

5

21 2

0

4

0

3

00

4

0

14

1

6

10

8

24

13

Nu

mb

er

of

err

os

Overview of medical errors made at post-intervention simulation

Intervention Group Control Group

20

Figure 12 – Change in distractor management between baseline and post-intervention simulations

0 5 10 15 20 25 30 35 40

Intervention group

Control group

4

37

29

38

Number of distractions mishandled

Change in distractor management between baseline and post-intervention simulations

Baseline

Post-intervention

21

Students completed an online electronic questionnaire on their experience of the simulated

ward round. 27 out of 28 students completed the evaluation (response rate = 96.43%). The

simulations were deemed to be valuable and of high-fidelity. All students felt that mandatory

integration of this experience into the final year curriculum to be important.

22

DISCUSSION

Final year medical students are not equipped with the skills to manage ward-based

distraction, in order to mitigate common medical error. This research is one of the first

studies of its kind to demonstrate that simulated ward training can improve undergraduate

patient safety behaviours.

The study suggests that didactic patient safety teaching is of limited benefit to

undergraduates. All students at the start of the academic year (and prior to study

commencement) received didactic teaching on a range of patient safety domains: including

non-technical skills and dealing with stress at work. Despite such instruction, students

committed large numbers of medical errors at baseline simulation.

Study limitations

• The study was conducted a small single-centre trial with 14 participants in each arm.

Despite the small sample size, study power was over 80% owing to the magnitude of

error-reduction rates.

• Despite randomisation the author accepts the potential for selection bias. Participants

took part on a voluntary basis – and arguably an engaged volunteer cohort, may be most

amenable to the effects of simulation and feedback.

• This study was not designed as a randomised control trial. This could have been achieved

by recruiting all participants at the same time in the academic calendar. However, all

students share on-site accommodation. As such, there was concern about cross-

contamination.

• Feedback delivered to the intervention group focussed on both distractor management and

medical error. Future studies should include additional intervention arms. For example,

feedback on distractor management alone.

23

RECOMMENDATION

Consideration to curricular integration of simulated ward experience is recommended. The

cost of running a single day of simulation, as described, per student is £61 (which includes

consumables and faculty cost). However, modalities to minimise cost and reduce faculty

burden to potentiate feasibility exist. For research purposes the simulated ward round

experience was faculty-intensive. Pragmatically, this experience could be resourced by three

volunteer patients, one staff nurse and a single clinical teaching fellow. To increase student

throughput, having a two-patient ward round would minimise simulation duration to 20

minutes, and increase daily capacity to 17 students. Revised costings for the faculty-light

simulation are £27.52 per student.

The following three areas of future research are suggested:

• Evidencing a curriculum-feasible version of the simulation, suitable for institutional roll-

out.

• Development and assessment of this teaching modality as a vehicle for inter-professional

education.

• To assess the predictive validity of a simulated ward round in assessing future

performance. For example, do simulated ward round behaviours transfer into clinical

practice?

24

CONCLUSION

The development of the experience comes at a pivotal time in medical education: in wake of

the recent Francis 36 report – an era that will see patient safety teaching in undergraduate

curriculums come under much sharper focus. Incorporation of simulation-based education

into undergraduate institutions is challenging. However curricular integration as outlined by

best practice in simulation 37 is essential if the benefits of simulation are to be realised. Given

student acceptability and positive patient safety behavioural change, curricular integration is a

challenge that medical educators must now meet - as pragmatic solutions to potentiate

feasibility exist.

Word count = 1500 words

(Excluding abstract and references)

25

REFERENCES

Citation Reference

1 KOHN, L. et al., 2000.

To err is Human – Building a safer health system.

1st Edition. National Academy Press. Washington, D.C.

2 NULL, G. et al., 2003.

Death by medicine

http://www.webdc.com/pdfs/deathbymedicine.pdf (Accessed 05.10.13)

3 SCHIMMEL, E., 1964.

The hazards of hospitalization. Annals of Internal Medicine, 60: pp. 100-110

4 STEEL, K. et al., 1981.

Iatrogenic illness on a general medical service at a university hospital. New

England Journal of Medicine, 304: pp. 638-642

5 BEDELL, S. et al., 1991.

Incidence and characteristics of preventable iatrogenic cardiac arrest. Journal

of the American Medical Association, 265(21), pp. 2815-2820

6 FLIN, R. et al., 2008.

Safety at the Sharp End: A guide to non-technical skills. Aldershot: Ashgate

Publishing.

7 SCHAREIN, P. and TRENDELENBURG, M., 2013.

Critical incidents in a tertiary care clinic for internal medicine. Biomed Central

Research Notes, 16(6), pp. 276

8 COOPER, J. et al., 1978.

Preventable anaesthesia mishaps: a study of human factors. Anaesthesiology,

49(6), pp. 399-406

9 SPENCER, R. et al., 2004.

Variation in communication loads on clinical staff in the emergency

department. Annals of Emergency Medicine, 44(3), pp. 268-273

10 TIPPING, M. et al., 2010.

Where did the day go? A time-motion study of hospitalists. Journal of Hospital

Medicine, 5(6), pp. 323-328

11 WEIGL, M. et al., 2013.

Relationship of multitasking, physicians’ strain and performance: an

observation study in ward physicians. Journal of Patient Safety, 9(1), pp. 18-23

12 WEIGL, M. et al., 2011.

Hospital doctors’ workflow interruptions and activities: an observation study.

British Medical Journal of Quality and Safety, 20(6), pp. 491-497

13 HOUSTON, D. et al., 1997.

Psychological distress and error making among junior house officers. British

Journal of Health Psychology, 2(2), pp. 141–151.

14 FIRTH-COZENS, J. et al., 1997.

Doctors’ perceptions of the links between stress and lowered clinical care.

Social Science and Medicine, 44(7), pp. 1017-1022

15 FEDDOCK, C. et al., 2007.

Do pressure and fatigue influence resident job performance? Medical Teacher,

29(5), pp.495-497

26

16 WESTBROOK, J. et al., 2010.

Association of interruptions with an increased risk and severity of medication

administration errors.

Archives of Internal Medicine, 170(8), pp. 683-690

17 MOORTHY, K. et al., 2003.

The effect of stress-inducing conditions on the performance of a laparoscopic

task. Surgical Endoscopy, 17(9), pp. 1481-1484

18 STUCKY, E. et al., 2009.

Intern to attending: assessing stress among physicians. Academic Medicine,

84(2): 251-257

19 ARORA, S. et al., 2010.

The impact of stress on surgical performance: a systematic review of the

literature. Surgery, 147(3), pp. 318-330

20 PERSOON, M. et al., 2011

The effect of distraction in the operating room during endourological

procedures. Surgical Endoscopy, 25(2), pp. 437-443

21 ILLING, J. et al., 2008.

How prepared are medical graduates to begin practice? A comparison of three

diverse UK medical schools.

The General Medical Council.

http://www.gmcuk.org/FINAL_How_prepared_are_medical_

graduates_to_begin_practice_September_08.pdf_29697834.pdf

(Accessed 15.09.13)

22 WORLD HEALTH ORGANISATION, 2009

WHO Patient Safety Curriculum Guide. World Health Organisation.

http://www.who.int/patientsafety/education/curriculum/en/

(Accessed 15.09.13)

23 GENERAL MEDICAL COUNCIL, 2009.

Tomorrow’s Doctors. GMC.

http://www.gmcuk.org/TomorrowsDoctors_2009.pdf_39260971.pdf (Accessed 18.10.13)

24 LIU, D. et al., 2009.

Interruptions and blood transfusion checks: lessons from the simulated

operating room.

Anesthesia and Analgesia, 108(1), pp. 219-222

25 MERRY, A. et al., 2008.

A simulation design for research evaluating safety innovations in anaesthesia.

Anaesthesia, 63(12), pp. 1349-1357

26 HARVEY, A. et al, 2012.

Impact of stress on resident performance in simulated trauma scenarios. Journal

of Trauma and Acute Care Surgery, 72(2), pp. 497-503

27 POTTIER, P. et al., 2013.

Effect of stress on clinical reasoning during simulated ambulatory consultations.

Medical Teacher, 35(6), pp. 472-480

28 LEBLANC, V. et al., 2005.

Paramedic performance in calculating drug doses following stressful scenarios

in a human patient simulator. Pre-hospital Emergency Care, 9(4), pp. 439-444

29 NEALE, G. et al., 2001.

Exploring the causes of adverse events in NHS hospital practice. Journal of the

27

Royal Society of Medicine, 94(7), pp. 322–330

30 NIKENDI, C. et al., 2008.

Ward rounds: how prepared are future doctors? Medical Teacher, 30(1), pp.

88-91

31 MCGLYNN, M. et al., 2012.

How we equip undergraduates with prioritisation skills using simulated

teaching scenarios. Medical Teacher, 34(7), pp. 526-529

32 MCGREGOR, C. et al., 2012.

Preparing medical students for clinical decision making: a pilot study exploring

how students make decisions and the perceived impact of a clinical decision

making teaching intervention. Medical Teacher, 34(7), pp 508-517

33 SMITH, S. et al., 2012.

A study of innovative patient safety education. Clinical Teacher, 9(1), pp. 37-

40

34 PUCHER, P. et al., 2013.

Validation of the Simulated Ward Environment for Assessment of Ward-Based

Surgical Care.

Annals of Surgery, March 6th 2013. (E-pub ahead of print)

35 SILVERMAN, J. et al., 1997.

The Calgary-Cambridge approach to communication skills teaching: The SET-

GO method of descriptive feedback.

Education for General Practice, 8: pp. 16-23

36 FRANCIS, R., 2013.

The Francis Enquiry. The Health Foundation.

http://www.health.org.uk/areas-of-work/francis-inquiry

(Accessed 12.12.13)27

37 ISSENBERG, S. et al., 2005.

Features and uses of high-fidelity medical simulations that lead to effective

learning: a BEME systematic review. Medical Teacher, 27(1), pp. 10-28.