Human Error Lecture

Anna L Cox | University College London | [email protected]

Human Error


overview

• a basic taxonomy of human error• theory: memory for goals• factors that increase error• reducing error


Why study human error?

• practically: to avoid catastrophic accidents.

• theoretically: to predict when one errs.•Cog Sci: it’s the same system that

produces correct behaviour most of the time! Tells us a lot about human cognition.


Beatson Oncology Case • LN, age 15• chemotherapy & 19 of 20 or 21 planned

radiotherapy treatments•Because the tumor type, location, and

extent and the patient size, age, and medical condition vary, the treatment for each patient is unique

• Each dose 58% HIGHER than it should have been….and no one noticed

• Scaling factor wasn’t entered


Blame• "the error ... was procedural and was not associated in any way with faults or deficiencies in the Varis 7 computer system" (Johnston, 2006; ii).

• “... at no point in the investigation was it deemed necessary to discuss the incident with the suppliers if this equipment since there was no suggestion that these products contributed to the error.” (Johnston, 2006; 2)


Accidents due to Human Errors

0%

20%

40%

60%

80%

100%

Petrochemicalplants

Medicine Worldwide Jetcargo

US nuclearpower plants

Automobiles Air trafficcontrol

Human Error Others


Annual Death Rates in US

329 395914986

43649

98000

0

50000

100000

commericalaviationdeaths

drowningdeaths

deaths fromfalls

motor vehicledeaths

deaths frommedicalerrors

One jumbo jet crash every day


Denise Melanson

• infusion pump containing a four day dose of two chemotherapy drugs, 5-fluorouracil and cisplatin, to administer to herself at home.

• pump dispensed the dose in four hours and not four days


Blame

• The nurse incorrectly calculated the dose•A second nurse checked the calculation

but didn’t notice the error• The drug bottle first displayed the per

24hr dose, (and then, in brackets the per 1hr dose)


OptiClik


JB’s video

http://www.youtube.com/watch?v=Hm7k0TRaPHI


Human Errors

Slips

A system fails to achieve theintended outcome in a planned sequence

of mental or physical activities

Incorrect execution of a correct action sequence

Correct execution of an incorrect action sequence

Mistakes


Human Errors

Slips Mistakes

ExecutionSlips

EvaluationSlips

KnowledgeBased

RuleBased

• ---------• ---------• ---------

• ---------• ---------• ---------

• ---------• ---------• ---------

• ---------• ---------• ---------


System Activities

User Activities

Physical System

Goal

Delete file

IntentionUse delete key

ActionSpecification

Select file + hit delete”

ExecutionUse mouse and keyboard

to perform action

sequence

Evaluation Form sub-goal

Interpretation Nothing happened

Perception Screen doesn’t change

Norman’s Action Theory


Mistakes

User Activities

System Activities

Goal

Physical System

Execution

SpecificationAction

Intention Evaluation

Interpretation

Perception


Mistakes

Knowledge-Based Rule-Based

• Faulty conceptual knowledge• Incomplete knowledge

• Biases and faulty heuristics• Incorrect selection of knowledge

• Information overload

• Misapplication of good rules

• Encoding deficiencies in rules

• Action deficiencies in rules

• Dissociation between knowledge and rules


Slips

User Activities

System Activities

Goal

Interruptions

Execution Slips Evaluation Slips

Intention

SpecificationAction

Execution

Physical System

Evaluation

Interpretation

Perception


“I intended to pick up my prescription on my way home. I

drove home directly.”

Capture Slip: automatic activation of a well-learned routine that overrides the current intended activity .


Double Capture Slip: The unintended activation of a related strong action schema.

“I meant to take off only my shoes, but took my socks off

as well.”


Omission Slip: due to interruptions.


Loss of Activation Slip


Description Slip: incomplete or ambiguous specification of intention that is similar to a familiar intention.


Associative Activation Slip: activation of similar but incorrect schemas.


Perceptual Confusion Slip


Repetition of Action Slip: repetition of an correctly performed action.

A nurse repeated radiation therapy

to a patient three times in a row, due to poor feedback. The patient died

three months later.


Cross-talk Slip (concurrent): action components are exchanged between two or more concurrent actions.

Physical Systems

Goal

Intention

ActionSpecification

Execution

Evaluation

Interpretation

Perception

Physical Systems

Goal

Intention

ActionSpecification

Execution

Evaluation

Interpretation

Perception

English writing Dutch reading


Cross-talk Slip (sequential): action components are exchanged between two or more sequential actions.

Physical Systems

Goal

Intention

ActionSpecification

Execution

Evaluation

Interpretation

Perception

Physical Systems

Goal

Intention

ActionSpecification

Execution

Evaluation

Interpretation

Perception

“I had just finished talking on the phone when my secretary ushered in some visitors. I got up from

behind the desk and walked to greet them with my hand outstretched saying ‘Smith speaking’.”

Talking on the phone (previous) Greeting (current)


Applying Norman’s Action Cycle to Error Classification

• Errors are sometimes made when programming infusion pumps

• These errors might occur when entering numbers

•What causes these errors and when to they occur?


Number Entry Error Taxonomy


#errordiary

•We’re tweeting when we make an error•Have a go at classifying these errors

using Norman’s action cycle• Tweet when you make an error


issues•Norman relied on self-report data of errors- gives a sense of common everyday errors- no reliable estimate of frequency of error- cannot determine cause of error

•move to the lab- difficulty is that errors are normally infrequent- this brings about issue for how to operationalize design and conduct statistical analysis - self-monitoring- slip errors occur with routine, procedural tasks - critical to train participants to criterion


post-completion error• routine, procedural task- goal-directed task- many subtasks

• post-completion step- the final step in a procedure- completed after the goal is achieved

• post-completion error (PCE)- missing the post-completion step- US ATM, photocopying, filling petrol, etc


example: photocopying


memory for goals

• goals are declarative memory representations defined by activation• most active goal directs behavior• activation declines over time, rehearsal can strengthen goal activation• associative links between goals propagate activation through network• completing ‘make copies’ lessens activation of ‘remove original’

Time

Threshold


Working memory capacity & workload


experiment•memory for goals predicts that working memory load should mediate likelihood of error

• seminal study: Byrne & Bovair (1997) - working memory capacity: with larger capacity the goal can be actively rehearsed, making error less likely- memory load: with increased load the main goal will decay more rapidly so high chance of error because post-completion step less likely to reach activation threshold


Byrne & Bovair’s results


Interruptions


Interruptions in the real world• Nurses are frequently

interrupted during their duties

• 20% of the time they don’t go back to the task they were working on when they were interrupted

• Sometimes it’s because that task is redundant

• Sometimes it’s because they’ve forgotten

• What are the implications for patient care?


•Could argue that she wasn’t paying attention

• She was blamed for not looking at the device

•But what about when we do direct our attention to the device again

•Does an interruption have any effect on what happens next?


effect of interruption• Li et al. (2008): multi-step procedural task making doughnuts with purposeful post-completion step

• interrupted to switch tasks to pack doughnuts


effect of interruptioner

ror r

ate

Interruption position


Reducing errors (I)

• cues


reducing errors

• system redesign- post-completion errors can often be designed out - e.g., ATM: card then cash vs. cash then card- but not always feasible for large complex systems

• provide explicit cues- must be visually salient, just-in-time, and meaningful- cues that are not specific are ineffective- habituation is a concern


visual cueing: Chung& Byrne (2008)


visual cueing, revisited


cue effectiveness


But…

•Not necessarily the perfect design solution

• The ‘just-in-time’ aspect can make it hard to design in

•And people become habituated to cues that are in the world all the time and so they stop working


Reducing errors (II)

• Slow down!


effect of interruptionEr

ror

Rate

(%

)

Interruption Positiondata from Li, S.Y.W., Blandford, A., Cairns, P., & Young, R.M. (2008). Journal of Experimental Psychology: Applied, 14, 314-328.


distribution of errors

•working on routine task

•interruption occurs in between subtasks

•on resumption most common error is to repeat or skip a step

•this is because of competition between goals at time of retrieval

data from Trafton, J. G., Altmann, E. M., & Ratwani, R. M., (2009). A memory for goals model of sequential action. International Conference of Cognitive Modeling, 2009.


Speed-accuracy tradeoff• some errors are caused by failure of memory

•memory is sensitive to changes in speed-accuracy tradeoff criterion

• error rate should be reduced if people invest time in thinking about where they were before resuming

•we use an enforced lockout procedure


lockout reduces error rateEr

ror

rate

(%

)

Condition


interruption-only trialsFr

eque

ncy

Resumption time (seconds)


find out more


Lockouts in design?

•What would YOU do if you were locked out of a system after an interruption?


implications for design• errors will occur, even with skilled users• errors reduced by good system design- well structured tasks- few interruptions- low memory demand- easy to select buttons- salient display (particularly mode indicators)- certain visual cues can be useful

• aid recovery from errors- make actions reversible- make the results of each action apparent


Summary

• Slip errors occur infrequently – but are persistent• Increasing speed => increasing error rate• Interruptions => increasing error rate

• …err what’s it like on a hosptial ward??

• Cues: have to be “just-in-time” and very aggressive

• Take your time – slow down!


reading• core

- Byrne, M. D., & Bovair, S. (1997). A working memory model of a common procedural error. Cognitive Science, 21, 31-61.- Norman, D.A. (1981). Categorization of action slips. Psychological Review, 88, 1-15.

• supplementary- Chung, P.H., & Byrne, M.D. (2008). Cue effectiveness in mitigating

postcompletion errors in a routine procedural task. International Journal of Human-Computer Studies, 66, 217-232. - Li, S.Y.W., Blandford, A., Cairns, P., & Young, R.M. (2008). The Effect of interruptions on postcompletion and other procedural errors: An account based on the activation-based goal memory model. Journal of Experimental Psychology: Applied, 14, 314-328.- Ratwani, R.M., McCurry, J.M., & Trafton, J.G. (2008). Predicting postcompletion errors using eye movements. In Proceedings of CHI’08 (pp. 539-542). New York, NY: ACM Press.- Reason, J. (1990). Human Error. Cambridge University Press.


Microwave Racing video

http://www.youtube.com/watch?v=Bzy5hVvbei8

Human Error Lecture

Education

Transcript of Human Error Lecture