Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making...

20
Computer Science for Fun Issue 17 ISSN 1754-3657 (Print) ISSN 1754-3665 (Online) Machines Making Medicine Safer Become an elite wizard Nurses in the mist Pit-stop heart surgery

Transcript of Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making...

Page 1: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

Computer Science for Fun Issue 17

ISSN 1754-3657 (Print)ISSN 1754-3665 (Online)

MachinesMakingMedicineSafer

Become an elitewizard

Nurses in the mist

Pit-stop heartsurgery

Page 2: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

Computer scientists savelives. Our hospitals arefull of computers indisguise, helping medicskeep patients alive.

We explore the world of safemedical machines through aLondon-Swansea research project‘CHI+MED’. Computer scientists areworking with psychologists, socialscientists and medical professionalsto make medical devices even safer.

To build machines that not onlysave lives but are safe you need tothink about the bigger picture, notjust the gadgets. Programming themis important but it is only a smallpart of what matters. Only when youknow how they are really used canyou find ways to make them evensafer.

We will also see how a Formula 1team have contributed to saferoperations, how taking a leaf out ofa gorilla watcher’s book has helpedone device manufacturer get amarket edge, and why programmersreally are wizards!

MachinesMakingMedicineSafer

www.cs4fn.org

It's rare for softwareproblems in medicaldevices to harmpeople, but it has

happened. Writing programscan be fun but it’s serious too. Back in the 1980s a

new radiation therapy machinecalled Therac-25 was released. It was an improvement on an earlier machine. They were designed to zap cancerwith a beam of radiation. Radiation is dangerous, whichmakes that risky business. But, with the right dose, it cancure cancer! Therapy machines include safety systems to ensure patients aren’taccidentally given an overdose. The earlier Therac-20 had a mechanicalsystem that made it physically impossible for a high-powered beam to bedirected at the patient. For the new model the physical system was replacedby software - ironically because it was thought to be safer and more advanced.It could produce a low energy beam of electrons, but also a high energy beamof x-rays by firing high-powered electrons at a metal plate.

Unfortunately, because of a bug - a programming mistake - it was sometimespossible for the complex software to get the timing off and do things in the wrong order - this is known as a ‘race condition’. They are really hard bugs to find. It meant that the high powered beam could be fired without the protective metal guard being in place and the patient could be given an overdose. It turned out that the operating system software used by themachines had been created by a programmer without proper training. It should never have been used in a device where safety mattered.

At least some good has come from it. It showed that safety critical softwarehas to be designed to be more than just safe. It has to be designed so you cancheck that it really is safe. Therac-25 also highlighted how important it is thatall programmers writing software used in systems that could harm people aretrained in safety-critical code development, understand safety and useappropriate methods to write and check code.

Deadlybugs

Welcome to cs4fn

2

Page 3: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

3

ProgrammingWizards

A magic trick is made of two parts:a secret method and a way ofpresenting it. Unless both work, the magic trick will fall flat.Software is similar. It also consistsof two things. The first is analgorithm: the instructions that whenfollowed by the computer lead to it doing whatever it’s there to do.The second is the human-computerinterface: the code that makes iteasy to use. It turns out algorithmsare just like magic secret methodsand interfaces are like thepresentation of a trick.

Many magic tricks are ‘self-working’. That means they don’t involve sleight ofhand or hidden mechanisms. As long asyou follow the steps the magical effectoccurs and your audience is leftsurprised. But that is what computerscientists mean by an ‘algorithm’.Programs are just algorithms written down in a language a computer can

follow. Self-working tricks are algorithmsfor doing magic. In fact the algorithms of some tricks are based on ones actually used in computing applications.For example, there are powerful tricksbased on the algorithm used by earlycomputers to search for data.

Most tricks have an underlying algorithm,but also include steps that rely on somekind of sleight of hand or misdirection.The audience look in the wrong place atthe wrong time, or are made to believesomething that isn’t true. They don’t seeeverything, even if everything is there tobe seen. The magician engineers themagic ‘system’ so that the whole audiencemake the same mistake at the same time.

It is of course similarly possible toengineer a computer system so that its users make mistakes too. Unlike amagician programmers don’t do it onpurpose. Some human errors are not aboutnegligence or stupidity but are about thelimitations of our brains and the way wesee the world. That is exactly what amagician relies on. Software developershave to do the opposite. They have todesign the system so that no one makesmistakes in using them. If they don’t takehuman limitations into account theirprograms will mislead.

Misdirection, is one way a magicianmisleads. It relies on the fact that we canonly focus our attention on one small areaat a time. If it is drawn to one point thenwe will miss other things. Instead ofdrawing a person’s attention away fromthings that matter, with software, theirattention should be drawn to those criticalthings. For example, if a nurse mistypes a dose into a machine delivering a drug,then before starting the infusion, youwant the nurse's eye drawn back to thescreen, not away from it, using themagician's tricks, so they check the dose is as expected.

Whether designing an algorithm or aninterface, the principles are the same as those behind creating a great magictrick. When programmers are writing new programs they are using the samecomputational thinking skills as magiciansinventing new tricks. Learn to programand you could become a real wizard!

Try a magic trick that relies onmisdirection for yourself and see how easily people make mistakes. Go to Magazine+ on the cs4fn websitewww.cs4fn.org and look for the FourAces trick.

Elite programmers are often called‘wizards’. It turns out that isn’t farfrom the truth!

The whole audiencemake the same mistakeat the same time

Both programmers and magicians rely on misdirection

cs4fn�eecs.qmul.ac.uk

Page 4: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

A red sock in with your white clothes wash – guess what happenednext? What can you do to prevent it from happening again? Whyshould a computer scientist care? Jo Brodie investigates. It turns outthat red socks have something to teach us about medical gadgets.

4 www.cs4fn.org

a problem that systems thinking cansolve. That way we can find solutions thatwork for everyone. One possibility is tocheck whether changes to the devicemight make mistakes less likely in thefirst place.

Errors? Or arrows?

Most medical machines are controlled witha panel with numbered keys (a numberkeypad) like on mobile phones, or up anddown arrows (an arrow keypad) like yousometimes get on alarm clocks. CHI+MEDresearchers have been asking questionslike: which way is best for enteringnumbers quickly, but also which is best for entering numbers accurately? They’vebeen running experiments where peopleuse different keypads, are timed and theirmistakes are recorded. The researchersalso track where people are looking whilethey use the keypads. Another approachhas been to create mathematicaldescriptions of the different keypads andthen mathematically explore how baddifferent errors might be.

It turns out that if you can see thenumbers on a keypad in front of you it’svery easy to type them in quickly, thoughnot always correctly! You need to checkthe display to see if you have actually putin the right ones. Worse, mistakes that aremade are often massive - ten times toomuch or more. The arrow keypads are alittle slower to use but because people arealready looking at the display (to see whatnumbers are appearing) they can helpnurses be more accurate, not only arefewer mistakes made but those that aremade tend to be smaller.

The

ofred soc

How can we stop red socks from everturning our clothes pink again? We need a strategy. Here are some possibilities.

• Don’t wear red socks.

• Take a ‘how to wash your clothes’ course.

• Never make mistakes.

• Get used to pink clothes.

Let’s look at them in turn - will they work?

Don’t wear red socks: That might help butit’s not much use if you like red socks or if you need them to match your outfit.And how would it help when you wearpurple, blue or green socks? Perhaps your clothes will just turn green instead.

Take a ‘how to wash your clothes’ course:Training might help: you’d certainly learnthat a red sock and white clothesshouldn’t be mixed, you probably didknow that anyway, though. It won’t stopyou making a similar mistake again.

Never make misteaks: Just never leave a red sock in your white wash. If only!Unfortunately everyone makes mistakes -that’s why we have erasers on pencils anda delete key on computers - this idea justwon’t work.

Get used to pink clothes: Maybe, but it’snot ideal. It might not be so great turningup to school in a pink shirt.

What if the problem’smore serious?

We can probably live with pink clothes,but what happens if a similar mistake is made at a hospital? Not socks, butmedicines. We know everyone makesmistakes so how do we stop thosemistakes from harming patients? Specialmachines are used in hospitals to pumpmedicine directly into a patient’s arm, forexample, and a nurse needs to tell it howmuch medicine to give - if the dose iswrong the patient won’t get better, andmight even get worse.

What have we learned from our red sockstrategies? We can’t stop giving patientsmedicine and we don’t want to get usedto mistakes so our first and fourthstrategies won’t work. We can give nurses more training but everyone makesmistakes even when trained, so the thirdsuggestion isn’t good enough either and it doesn’t stop someone else making thesame mistake.

We need to stop thinking of mistakes as aproblem that people make and instead as

DOOM

Image credit - Flickr user DomFurniss

Page 5: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

sort of doses are likely and safe. Someonemight still mistakenly tell the machine togive too high a dose but now it can catchthe error and ask the nurse to double-check. That’s like having a washingmachine that can spot bright socks in awhite wash and that refuses to switch ontill it has been removed.

Building machines with a better ability to catch errors (remember, we all makemistakes) and helping users to recoverfrom them easily is much more reliablethan trying to get rid of all possible errorsby training people. It’s not about avoidingred socks, or errors, but about puttingbetter systems in place to make sure thatwe find them before we press that big‘Start’ button.

ck

Smart machines help users

A medical device that actively helps usersavoid mistakes helps everyone using it (andthe patients it’s being used on!). Changingthe interface to reduce errors isn’t the onlysolution though. Modern machines have‘intelligent drug libraries’ that containinformation about the medicines and what

M

cs4fn�eecs.qmul.ac.uk 5

Page 6: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

6 www.cs4fn.org

Just as the pit-stop team are underintense time pressure, the operatingtheatre team are under pressure to beback in the operating theatre for the next operation as soon as possible. In ahandover from surgery there is lots ofscope for small mistakes to be made thatslow things down or cause problems thatneed to be fixed. In situations like this,it’s not just the technology that mattersbut the way everyone works togetheraround it. The system as a whole needs tobe well designed and pit stop teams areclearly in the lead.

Smooth moves

To find out more, the research teamwatched the Ferrari F1 team practice pit-stops as well as talking to the race

Open-heart surgery is obviously acomplicated business. It involves a bigteam of people working with a lot oftechnology to do a complicated operation.Both during and after the operation thepatient is kept alive by computer: lots ofcomputers in fact. A ventilator isbreathing for them, other computers arepumping drugs through their veins andyet more are monitoring them so thedoctors know how their body is coping.

Pass it on

One of the critical times in open-heartsurgery is actually after it is all over. The patient has to be moved from theoperating theatre to the intensive careunit where a ‘handover’ happens. All themachines they were connected to have to

be removed, moved with them or swappedfor those in the intensive care unit. Notonly that, a lot of information has to bepassed from the operating team to thecare team. The team taking over need to know the important details of whathappened and especially any problems, if they are to give the best care possible.

A research team from the University ofOxford and Great Ormond Street Hospitalin London wondered if hospital teamscould learn anything from the way othercritical teams work. This is an importantpart of computational thinking – the waycomputer scientists solve problems. Ratherthan starting from scratch, find a similarproblem that has already been solved andadapt its solution for the new situation.

Pit-stop hea The Formula 1 car screams to a stop in the pit-lane. Seven seconds later,it has roared away again, back into the race. In those few seconds it hasbeen refuelled and all four wheels changed. Formula 1 pit-stops are theultimate in high-tech team work. Now the Ferrari pit stop team havehelped improve the hospital care of children after open-heart surgery!

Page 7: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

These and other changes led to what the researchers hoped would be a much improved way of doing handovers.But was it better?

Calm efficiency saves the day

To find out they studied 50 handovers –roughly half before the change was madeand half after. That way they had a directway of seeing the difference. They used a checklist of common problems notingboth mistakes made and steps thatproved unusually difficult. They alsonoted how well the teams workedtogether: whether they were calm andsupported each other, planned what theydid, whether equipment was availablewhen needed, and so on.

They found that the changes led to clearlybetter handovers. Fewer errors were madeboth with the technology and in passing on information. Better still, while the bestperformance still happened when theteams worked well, the changes meant thatteamwork problems became less critical.

Pit-stops and open-heart surgery may be aworld apart, with one being about getting

every last millisecond of speed and theother about giving as good care aspossible. But if you want to improve howwell technology and people work together,you need to think about more than justthe gadgets. It is worth looking forsolutions anywhere: children can behelped to recover from heart surgery evenby the high-octane glitz of Formula 1.

director about how they worked. Theythen talked to operating theatre andintensive care unit teams to see how theideas might work in a hospital handover.They came up with lots of changes to theway the hospital did the handover.

For example, in a pit-stop there is oneperson coordinating everything – theperson with the ‘lollipop’ sign thatreminds the driver to keep their brakeson. In the hospital handover there was no person with that job. In the newversion the anaesthetist was given theoverall job for coordinating the team.Once the handover was completed thatresponsibility was formally passed to theintensive care unit doctor. In Formula 1each person has only one or two cleartasks to do. In the hospital people’s roleswere less obvious. So each person wasgiven a clear responsibility: the nurseswere made responsible for issues with draining fluids from the patient,anaesthetist for ventilation issues, and soon. In Formula 1 checklists are used toavoid people missing steps. Nothing likethat was used in the handover so achecklist was created, to be used by the team taking on the patient.

art surgery

cs4fn�eecs.qmul.ac.uk 7

Image credit: Flickr user rafa_uoc

Page 8: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

www.cs4fn.org8

The nurse types in a dose of 100.1 mg of a powerful drug and pressesstart. It duly injects 1001 mg into the patient without telling the nurse that it didn’t do what it was told. You wouldn’t want to be that patient!

Much ado about

nothing

continue typing digits. Worse still thedevice ignores that decimal point silently.It doesn’t make any attempt to help anurse notice the change. A busy nursewould need to be extremely vigilant to seethe tiny decimal point was missing giventhe lack of warning.

A useful thing about Paolo’s approach isthat it gives you the button presses thatlead to the problem. With that you cancheck other devices very quickly. Hefound that medical devices from threeother manufacturers had exactly the same problem. Different teams had allprogrammed in the same problem. Nonehad thought that if their code ignored adecimal point, it ought to warn the nurseabout it rather than create a number tentimes bigger. It turns out that differentprogrammers are likely to think the sameway and so make the same mistakes (see‘Double or Nothing’, right).

Now the problem is known, nurses can be warned to be extra careful and themanufacturers can update the software.Better still they and the regulators nowhave an easy way to check theirprogrammers haven’t made the samemistake in future devices. In future,whether vigilant or not, a nurse won’t be able to get it wrong.

Designing a medical device is difficult.It’s not creating the physical machine thatcauses problems so much as writing thesoftware that controls everything that thatmachine does. The software is complexand it has to be right. But what do wemean by “right”? The most obvious thingis that when a nurse sets it to dosomething, that is exactly what it does.

Getting it right is subtler than thatthough. It must also be easy to use andnot mislead the nurse: the human-computer interface has to be right too. Itis the software that allows you to interactwith a gadget – what buttons you pressto get things done and what feedback youare given. There are some basic principlesto follow when designing interfaces. Oneis that the person using it should alwaysbe clearly told what it is doing.

Manufacturers need ways to check theirdevices meet these principles: to knowthat they got it right.

It’s not just the manufacturers, though.Regulators have the job of checking thatmachines that might harm people are‘right’ before they allow them to be sold.That’s really difficult given the softwarecould be millions of lines long. Worse theyonly have a short time to give an answer.

Problems may only happen once in amillion times a device is used. They arevirtually impossible to find by havingsomeone try possibilities to see whathappens, the traditional way software ischecked. Of course, if a million devicesare bought, then a million to one chancewill happen to someone, somewherealmost immediately!

Paolo Masci at Queen Mary University ofLondon, has come up with a way to helpand in doing so found a curious problem.He’s been working with the US regulatorfor medical devices – the FDA – anddeveloped a way to use maths to findproblems. It involves creating amathematical description of what criticalparts of the interface program do.Properties, like the user always knowingwhat is going on, can then be checkedusing maths. Paolo tried it out on thecode for entering numbers of a realmedical device and found some subtleproblems. He showed that if you typed in certain numbers, the machine actuallytreated them as a number ten timesbigger. Type in a dose of 100.1 and themachine really did set the dose to be1001. It ignored the decimal pointbecause on such a large dose it assumedsmall fractions are irrelevant. Howeveranother part of the code allows you to

Mathematical tools can be hard to use.Paolo with Enrico D’Urso from QueenMary and Patrick Oladimeji fromSwansea have developed a front-endthat allows interfaces to be easilycreated and tested behind the scenes.

Million to one chancesare guaranteed tohappen.

Page 9: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

9

on what to do. Then as long as morethan half agree on the right answerthe system as a whole will do theright thing. That’s the theory anyway.Unfortunately in practice it doesn’talways work. NancyLeveson, an expert in software safetyfrom MIT, ran an experiment where differentprogrammers weregiven programs towrite. She foundthey wrote code that gave the samewrong answers. Even if it had usedindependentlywritten redundantcode it’s stillpossible Ariane 5would have exploded.

Redundancy is a big help but it can’tguarantee softwareworks correctly. Whendesigning systems tobe highly reliable youhave to assumethings will still gowrong. You must stillhave ways to checkfor problems and todeal with them sothat a mistake(whether by humanor machine) won’tturn into adisaster.

A powerful way to improve reliability is to use redundancy: double things up. A plane with four engines can keep flyingif one fails. Worried about a flat tyre? You carry a spare in the boot. Thesesituations are about making physical parts reliable. Most machines are acombination of hardware and softwarethough. What about software redundancy?

You can have spare copies of software too.Rather than a single version of a programyou can have several copies running ondifferent machines. If one program goeswrong another can take over. It would benice if it was that simple, but software is different to hardware. Two identicalprograms will fail in the same way at thesame time: they are both following thesame instructions so if one goes wrongthe other will too. That was vividly shownby the maiden flight of the Ariane 5rocket. Less than 40 seconds from launchthings went wrong. The problem was to dowith a big number that needed 64 bits ofstorage space to hold it. The program’sinstructions moved it to a storage placewith only 16 bits. With not enough space,the number was mangled to fit. That ledto calculations by its guidance systemgoing wrong. The rocket veered off courseand exploded. The program wasduplicated, but both versions were thesame so both agreed on the same wronganswers. Seven billion dollars went up insmoke.

Can you get round this? One solution is toget different teams to write programs todo the same thing. The separate teamsmay make mistakes but surely they won’tall get the same thing wrong! Run themon different machines and let them vote

Double or

nothing?If you spent billions of dollars on a gadget you’d probably like it to lastmore than a minute before it blows up. That’s what happened to aEuropean Space Agency rocket. How do you make sure the worst doesn’t happen to you? How do you make machines reliable?

cs4fn�eecs.qmul.ac.uk

Page 10: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

What do you do when your boss tells you "go and invent a new product"? Lock yourself away and stare out thewindow? Go for a walk, waiting for inspiration? Systemengineers Pat Baird and Katie Hansbro did some anthropology.

Nurses inthe Mist

Dian Fossey is perhaps the most famous anthropologist. She spentover a decade living in the jungle with gorillas so that she couldunderstand them in a way no one had done before. She started tosee what it was really like to be a gorilla, showing that their fierceKing Kong image was wrong and that they are actually gentle giants:social animals with individual personalities and strong family ties.Her book and film, 'Gorillas in the Mist', tells the story.

Pat and Katie work for Baxter Healthcare. They are responsible fordeveloping medical devices like the infusion pumps hospitals use to pump drugs into people to keep them alive or reduce their pain.Hospitals don't buy medical devices like we buy phones, of course.They aren't bought just because they have lots of sexy new features.Hospitals buy new medical devices if they solve real problems. They want solutions that save lives, or save money, and if possibleboth! To invent something new that sells you ideally need to solveproblems your competitors aren't even aware of. Challenged to come up with something new, Pat and Katie wondered if, given theequivalent was so productive for Dian Fossey, perhaps immersingthemselves in hospitals with nurses would give the advantage theircompany was after. Their idea was that understanding what it wasreally like to be a nurse would make a big difference to their abilityto design medical devices. That helped with the real problemsnurses had rather than those that the sales people said wereproblems. After all the sales people only talk to the managers, and the managers don't work on the wards. They were right.

Taking notes

They took a team on a 3-month hospital tour, talking to people,watching them do their jobs and keeping notes of everything. They noted things like the layout of rooms and how big they were,recorded the temperature, how noisy it was, how many flashinglights and so on. They spent a lot of time in the critical care wardswhere infusion pumps were used the most but they also went to lotsof other wards and found the pumps being used in other ways. Theydidn't just talk to nurses either. Patients are moved around to havescans or change wards, so they followed them, talking to the portersdoing the pushing. They observed the rooms where the devices werecleaned and stored. They looked for places where people were doingad hoc things like sticking post it note reminders on machines. That might be an opportunity for them to help. They looked at themachines around the pumps. That told them about opportunities formaking the devices fit into the bigger tasks the nurses were usingthem as part of.

www.cs4fn.org10

Page 11: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

cs4fn�eecs.qmul.ac.uk 11

pumps. How do you do that? Drag thembehind by the tubes? Maybe themanufacturers can design in a way tomake it easy. No one had ever botheredtalking to the porters before. After allthey are the low paid people, doing thegrunt jobs, expected to be invisible.Except they are important and theirproblems matter to patient safety.

The advantages didn't stop there, either.Because of all that measuring, thecompany had the raw data to createmodels of lots of different wardenvironments that all the team could use when designing. It meant they couldexplore in a virtual environment how wellintroducing new technology might fixproblems (or even see what problems it would cause).

All in all anthropology was a big success.It turns out observing the detail matters.It gives a commercial advantage, and allthat mundane knowledge of what reallygoes on allowed the designers to redesigntheir pumps to fix potential problems.That makes the machines more reliable,and saves money on repairs. It's better for everyone.

Talking to porters, observing cupboards,watching ambulance bays: sometimes it's the mundane things that make thedifference. To be a great systems designeryou have to deeply understand all thepeople and situations you are designingfor, not just the power users and thenormal situations. If you want toinnovate, like Pat and Katie, take a leaf out of Dian Fossey's book. Try anthropology.

So did Katie and Pat come up with a newproduct as their boss wanted? Yes. Theydeveloped a whole new service that isbringing in the money, but they did muchmore too. They showed that anthropologybrings lots of advantages for medicaldevice companies. One part of Pat's job, for example, is to troubleshoot whenhis customers are having problems. Hefound after the study that, because heunderstood so much more about howpumps were used, he could diagnoseproblems more easily. That saved timeand money for everyone. For example,touch screen pumps were being damaged.It was because when they were storedtogether on a shelf their clips werescratching the ones behind. They had also seen patients sitting outside in theambulance bays with their pumps for longperiods smoking. Not their problem, apartfrom it was Texas and the temperatureoutside was higher than the safe operatinglimit of the electronics. Hospitals don’tget that hot so no one imagined theremight be a problem. Now they knew.

Porters shouldn’t be missed

Pat and Katie also showed that to designa really good product you had to designfor people you might not even thinkabout, never mind talk to. By watchingthe porters they saw there was a problemwhen a patient was on lots of drugs eachwith its own pump. The porter pushingthe bed also had to pull along a gaggle of

The hot Texansummer was a problem

Page 12: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

www.cs4fn.org12

A weestory

Going, going, gone

Rob was heading home from London afterthe office Christmas party. Emma, a workfriend, offered him a lift home. He sat inthe middle of the back seat with a girlfrom work on either side. He still had acan of drink to finish, but maybe bringingit wasn't such a good idea. Not too longinto the journey he began to feel the needto wee! It wasn't that far so surely he'd beable to hold on, though.

Unfortunately, the feeling grew stronger.Beads of sweat formed on his brow as hefought to hold the pee back. The others inthe car were chatting and laughing. Hedidn't want to be a hassle and ask themto pull over, and besides they were on amotorway, so it'd be difficult anyway.Weighing everything up Rob thought thebest course of action would be just to leta little bit out, which would relieve thepain and buy him time to have a properwee when he got home. You can probablyguess the rest... he sat there and watchedhis plan go wrong as a dark wet patchgrew and grew. Once started nothingcould stop it.

The relief paled into insignificance as thegravity of the situation sank in. He was inhis colleague's car, sat between twopeople from work, and without a whimperhe had wet himself. However, this willsurprise you: after such a stupid mistakeRob now performed an amazing act ofgenius - he faked his elbow being pushedby one of the girls and dropped his can of drink into his lap. She apologisedprofusely, he told her not to worry. Hesmiled smugly as he finished the journey

not as the perpetrator of a pee crime butas the victim of an unfortunate spillage.

The knowledge

What can this tell us about the sciencebehind human error? Well this is anexample of a 'knowledge-based error'.They happen when a person has somewrong or missing knowledge about asituation. In this case Rob should haveknown that you can't just let a little bitout. How could he not know? Surely,everyone knows that if you're dying for apee, once you pop you ain't stopping untilit's finished! The trouble is somethingthat is obvious to one person can beunknown to another. If Rob had thecorrect knowledge he would have chosena different course of action. Unfortunatelyhe didn't.

Preventing this kind of problem is aboutmaking sure people are well-trained andhave all the knowledge they need to dothe job. This is also where learning from mistakes is vital. If you make aknowledge-based error and understandwhy it happened, you should never make it again. You now have the correctknowledge. Ideally, others should learnfrom your mistake too, though, if youdon't hide it like Rob did.

In work situations, once we know of aproblem, we can train people so they havethe knowledge to choose the right coursesof action. If rather than looking to blameor shame when someone makes a mistakewe look for lessons to learn, we canprevent the problem happening to anyone in the future.

Risky baths

This is especially important with medicaltechnology. Take the tragic case of thepatient who died because of taking a hotbath. They had a patch inserted undertheir skin that delivered the drug theyneeded in a low-hassle way. What theydidn't know was that a hot bath increaseshow much drug is delivered by the patch.If we raise awareness of this danger it willhopefully prevent others patients doingthe same. A tragic accident like this canalso lead to a review of wider issues thanjust the particular thing that patientdidn't know. For example, reviewing howpatients are told about this sort of safetyinformation more generally might lead toother kinds of accidents being avoided.

That patient knew nothing about the risksof taking a hot bath. But suppose they didknow. It's still entirely plausible that theycould take a hot bath having justforgotten that they shouldn't. Part of theadvantage of these patches is that theycan be implanted then virtually forgottenabout. What is supposed to be a blessingturns into a curse. You mustn't forgetabout the patch if you are thinking about a bath. This is a completelydifferent kind of error to a knowledge-based error. It's called a ‘slip error’ and it needs a different kind of solutionto prevent it. It needs innovative designnot just the spreading of knowledge, butthat's another wee story.

'Father Christmas needs a wee' is a classic book that teaches kids tocount. It's all about something that secretly fascinates: how close doother people get to wetting themselves? It's just a story but it inspiredDom Furniss of UCL to tell us some festive, but true, stories about hisfriends. Read on to learn something useful about avoiding nasty accidents.

For a wee storyabout slips, go towww.cs4fn.org

Page 13: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

cs4fn�eecs.qmul.ac.uk 13

Why not contribute to research

and log your own everyday errors.

Go to www.errordiary.org

Erorrdiary streams funny, frustrating and

fatal errors. Whether exotic or mundane

and everyday, it aims to raise awareness

about how errors are all around us, no

matter how clever we pretend to be.

Page 14: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

Two weeks later, James' colleague, Julia,is on duty. She makes a similar mistaketreating a different patient, Peter. Exceptthat she doesn't notice her mistake untilthe bag of insulin has emptied. Becauseit took so long to spot, Peter needsemergency treatment. It's touch-and-gofor a while, but luckily he recovers.

Julia reports the incident through thehospital's incident reporting system, so atleast it can be prevented from happeningagain. She is wracked with guilt formaking the mistake, but also hopesfervently that she won't be blamed and so punished for what happened.

www.cs4fn.org14

Was it his fault? Should he have beenmore careful? He didn't choose to put thesugar in a high cupboard with the flour.

Maybe it was my fault? I didn't choose toput the sugar there either. But I didn't tellanyone about the first time it happened. Ididn't move the sugar to a lower cupboardso it was easier to reach either. So maybeit was my fault after all? I knew it was aproblem, and I didn't do anything about it.

Now think about your local hospital.

James is a nurse, working in intensive care.Penny is really ill and is being given insulinby a machine that pumps it directly intoher vein. The insulin is causing a sideeffect though - a drop in blood potassiumlevel - and that is life threatening. They

don't have time to set up a second pump,so the doctor decides to stop the insulinfor a while and to give a dose of potassiumthrough a second tube controlled by thesame pump. James sets up the bag ofpotassium and carefully programs thepump to deliver it, then turns his attentionto his next task. A few minutes later, heglances at the pump again and realisesthat he forgot to release the clamp on thetube from the bag of potassium. Penny isstill receiving insulin, not the potassiumshe urgently needs. He quickly releases theclamp, and the potassium starts to flow. Anhour later, Penny's blood potassium levelsare pretty much back to normal: she's stillill, but out of danger. Phew! Good job henoticed in time and no-one else knowsabout the mistake!

Oh no!Notagain...

What a mess. There's flour all over the kitchen floor. A fortnight ago I opened the cupboard to get sugar for my hot chocolate. As I pulled out the sugar, it knocked against thebag of flour which was too close to the edge... Luckily the bagdidn't burst and I cleared it up quickly before anyone foundout. Now it's two weeks later and exactly the same thing justhappened to my brother. This time the bag did burst and itwent everywhere. Now he's in big trouble for being so clumsy!

Page 15: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

Don’t miss the nearmisses

Why did it happen? There are a wholebunch of problems that are nothing to dowith Julia. Why wasn't it standard practiceto always have a second pump set up forcritically ill patients in case suchemergency treatment is needed? Whycan't the pump detect which bag the fluidis being pumped from? Why isn't it reallyobvious whether the clamp is open orclosed? Why can’t the pump detect it. If the first incident - a 'near miss' - hadbeen reported perhaps some of theseproblems might have been spotted andfixed. How many other times has ithappened but not reported?

What can we learn from this? Onething is that there are lots of ways of setting up and using systems, and some may well make them safer.Another is that reporting "near misses"is really important. They are a valuablesource of learning that can alert otherpeople to mistakes they might makeand lead to a search for ways ofmaking the system safer - but only if people tell others about them.Reporting near-misses can helpprevent the same thing happeningagain.

The above was just a story, but it's based on an account of a realincident... one that has been reportedso it might just save lives in thefuture.

cs4fn�eecs.qmul.ac.uk 15

Report it! Alexis Lewis, at Swansea University, hasbeen exploring how best to design incidentreporting forms as part of her PhD. Lots of different forms are used in hospitalsacross the UK and she examined morethan 20. Many had features that wouldmake it harder than necessary for nursesand doctors to report incidents. Somefailed to ask about important facts andmany didn’t encourage feedback. It wasn’tclear how much detail or even what shouldbe reported. She is using the results todesign a new reporting form that avoidsthe problems and that can be built into asystem that encourages the reporting ofincidents so that hospital staff learn fromthe incidents that do happen.

Whodunnit?To stop different people making the samemistakes over and over again you need toknow why they happen. Incidentinvestigators try to work that out. They actlike detectives digging up and reviewingevidence. The difference is that they are lessinterested in whodunnit than in what led toit happening. One way they do that is calledroot cause analysis. You try and track backbeyond the immediate reason somethingwent wrong to the ultimate causes. Forexample, if a doctor gave someone thewrong drug, it might be that they misreadthe name or that the two drugs were storedtogether so the wrong one easily picked up.Once they know why an incident happened,investigators make recommendations abouthow these real causes can be prevented.Perhaps certain drugs should not be storedtogether, for example.

The methods used help the investigatorfocus on the things that matter. HuayiHuang, a PhD student at Queen Mary, has developed a new way to to do this.Rather than focus on causes, his methodconcentrates on the way information travels around - from person to person andbetween people and machines. At eachpoint the idea is to look at what safetymeasures are in place to make sure thatinformation doesn’t go wrong. Rather thanfocus on whodunnit, focus on why nothingstopped it happening!

Page 16: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

www.cs4fn.org16

MicrowaveRacing

When you go shopping for a new gadget like an MP3 player are you mostly wowed byits sleek looks, or drool over its long list of extra functionality? Do you then not usethose extra functions because you don't know how? Rather than just drooling, why not go to the races to help find a device you will actually use.

On your marks, getset… microwave

Take an everyday gadget like a microwave.They have been around a while, somanufacturers have had a long time toimprove their designs and make themeasy to use. You wouldn't expect there tobe problems would you? There are lots ofways a gadget can be harder to use thannecessary - more button presses maybe,lots of menus to get lost in, more special key sequences to forget, easyopportunities to make mistakes, noobvious feedback to tell you what it'sdoing... Just trying to do simple thingswith each alternative is one way to checkout how easy they are to use. How simpleis it to cook some peas with yourmicrowave? Could it be simpler? DomFurniss, a researcher at UCL decided tovideo some microwave racing as a fun way to find out.

Everyday devices still cause peopleproblems even when they are trying to doreally simple things. What’s clear fromMicrowave racing is that some really areeasier to use than others. Does it matter?Perhaps not if it's just an odd minutewasted here or there cooking dinner or if actually, despite your drooling in theshop, you don't really care that you neveruse any of those 'advanced' features.

Would it matter to you more if the devicewere in a hospital keeping a patient aliveand where a mistake could harm them?There are lots of gadgets like this:infusion pumps for example. They are the machines you are hooked up to in ahospital via tubes. They pump life-savingdrugs, nutrient rich solutions or extrafluids to keep you hydrated directly into

your body. If the nurse makes a mistakesetting the rate or volume then it couldmake you worse rather than better. Surely then you want the device to help the nurse get it right.

While the consequences are completelydifferent, the core thing you do in settingan infusion pump is actually very similarto setting a microwave - "set a number forthe volume of drug and another for therate to infuse it and hit start" versus "seta number for the power and another forthe cooking time, then hit start". Thesame types of design solutions (both goodand bad) crop up in both cases. Nurseshave to set such gadgets day in day out.In an intensive care unit, they will beusing several at a time with each patient.Do you really want to waste lots ofminutes of such a nurse's time, day in,day out? Do you want a nurse to easily be able to make mistakes?

User feedback

What the microwave racing video shows isthat the designers of gadgets can makethem trivially simple to use. They can alsomake them very hard to use if they focusmore on the looks and functions of thething than ease of use. Manufacturers ofdevices are only likely to take ease of useseriously if the people doing the buyingmake it clear that we care. Mostly we givethe impression that we want features sothat is what we get. Microwave racing maynot be the best way to do it, but next timeyou are out looking for a new gadgetcheck how easy it is to use before youbuy... especially if the gadget is aninfusion pump and you happen to be theperson placing orders for a hospital!

Go to www.cs4fn.org to see Microwaveracing in action and find out how thepros really evaluate gadgets.

Better design helpsavoid mistakes

Page 17: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

cs4fn�eecs.qmul.ac.uk 17

Screamingheadline kills!!!

If press and politicians are pressurisinghospitals to show they have donesomething, they may just sack the personwho made the mistake. They may thennot improve things meaning the samething could happen again if it was anaccident waiting to happen. Worse if we’retoo quick to blame and punish someone,other people will be reluctant to reporttheir mistakes, and without that sharingwe can’t learn from them. One of thereasons flying is so safe is that pilotsalways report ‘near misses’ knowing theywill be praised for doing so, rather thangetting into trouble. It’s far better to learnfrom mistakes where nothing really badhappens than wait for a tragedy.

Share mistakes tolearn from them

Chrystie Myketiak from Queen Mary isexploring whether the way a medicaltechnology story is reported makes adifference to how we think about it, andultimately what happens. She analysednews stories about three similar incidentsin the UK, America and Canada. Shewanted to see what the papers said, butalso how they said it. The press often

Pumpin Blood

Tunetrace is a popular app that turnsdrawings and photographs into a musicallight show. Now this award winning ideahas gone into showbiz. With WarnerRecords and pop band NoNoNo, a newTunetrace app has been released thatallows you to resequence the band’s track,Pumpin Blood, based on the photographor drawing you give it. The app extractsthe pattern of line crossings in thepicture. That pattern is treated as a sortof computer program. Depending onsecret rules it uses it to jump the track in beat steps to create new musicalmasterpieces. Why not have a play,download it for free fromwww.qappsonline.com/apps/pumpin-blood/

Playing Bridge, butnot as we know it

Clifton, Forth and Brooklyn are all famoussuspension bridges where, through a featof engineering greatness, the roadwayhangs from cables slung from sturdytowers. The Human Harp project createdby Di Mainstone, Artist in Residence atQueen Mary, involves attaching digitalsensors to bridge cables attached by linesto the performer’s clothing. As the bridgevibrates to traffic and people, and theperformer moves, the angle and length of the lines are measured and differentsounds produced. In effect human and bridge become one augmentedinstrument, making music mutually. FInd out more at www.humanharp.org

sensationalise stories but Chrystiefound that this didn’t always happen.Some news stories did imply that theperson who’d made the mistake wasthe problem (it’s rarely that simple!)but others were more careful tohighlight that they were busy peopleworking under stressful conditions andthat the mistakes only happenedbecause there were other problems.Regulations in Canada mean the mediacan’t report on specific details of astory while it is being investigated.Chrystie found that, in the incidentsshe looked at, that led to much morereasoned reporting. In that kind ofenvironment hospitals are more likelyto improve rather than just blame staff.How the hospital handled a case alsoaffected what was written - being openand honest about a problem is betterthan ignoring requests for commentand pretending there isn’t a problem.

As we’ve seen everyone makesmistakes (if you don’t believe that, the next time you’re at a magic show,make sure none of the tricks fool you!).Often mistakes happen because thesystem wasn’t able to preventthem. Rather than blame, retrainor sack someone its far better

to improve the system.That way something goodwill come of tragedies.

Most people in hospital get great treatment but if somethingdoes go wrong the victims often want something good to comeof it. They want to understand why it happened and be sure it won’t happen to anyone else. Medical mistakes can make a big news story though with screaming headlines vilifying those ‘responsible’. It may sell papers but it could also makethings worse.

Page 18: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

18 www.cs4fn.org

Think of a numberIf you add 1 to 1219 you get 1220, except sometimes you get 1210. What’s goingon? It all depends on what the numbers mean and what they’re doing. It happens toyou all the time and you probably don’t bat an eyelid.

A typical padlock will only open when thefour-digit combination is dialed in so thatthe digits line up correctly. Each disc ofdigits is separate from the other three and turning one doesn’t affect any of the others. That makes sense - when youchoose a combination you are picking 4completely separate digits.

Clocks go forward

With a typical digital clock on the otherhand the digits can be changed separatelybut they also act as if they are linked. As the time approaches 7am the hourdigit won’t change from ‘6’ until the twominute digits have reached 59 (the clocknow reads 6:59). At precisely 7 o’clockthe two digits on the right will say 00 butthe hour digit will also move on from 6 to7 so that the time reads 7:00. If you weresetting an alarm for 7am you might beglad that all three digits act together, and not just the minute ones at the end -otherwise it would be easy to accidentallyset your alarm for 6 o’clock instead of 7!

With the padlock when you add 1 to a combination 1219 you get thecombination 1210. With the clock,adding 1 to the time 12:19 you get12:20. You need to know what yournumbers mean and so how they work if you are to change them correctly.

Can you hack it

There are lots of ways to enter numbersinto gadgets, choosing the right one for the job at hand can make a bigdifference. Perhaps there are ways no onehas thought of before! At a hack day run

by Gerrit Niezen of Swansea University,the CHI+MED research team played withdifferent ways of entering numbers, anddeveloped a bunch of new ideas to tryout. For example, Sarah Wiseman fromUniversity College London turned threeSifteo Game Cubes, an interactive toy,into devices for entering numbers.Smaller than a matchbox the cubes havea clickable screen that displays games, or in this case numbers, and they’resensitive to movement and position, a bit like Wii game controllers. Sarahprogrammed two of the cubes to displayone number each and the third cube toact as an up/down button to change thenumbers. She also told the number cubesto behave as separate digits (like thepadlock) if the controller was placedabove or below them, or as linkednumbers (like the alarm clock) if thecontroller was placed to the left or right ofthe cubes. Knowing whether the numbersin your machine act separately or togethercan make the difference between an extrahour in bed but it also helps patients toget the right dose of medicine.

Go to the cs4fn website to watch Sarah’svideo about numbers on YouTube.

credit: Flickr user IndieCade

Page 19: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

cs4fn�eecs.qmul.ac.uk 19

Many hospitals now use mobile devices totreat patients. Machines that are smalland light enough to move around with apatient mean that those patients don’thave to be stuck in bed. Some can eventake them home and have their treatmentthere: a much nicer experience. But thiscomes with some risks. Because mobiledevices can move they can also end upupside down, and numbers can go wrongwhen they’re the wrong way up.

If you’ve got a device strapped to yourarm then someone else looking at it willsee the numbers correctly but if you lookat it you’ll read them upside-down. If youlook at them in the mirror you’ll see thenumbers back to front. What numberslook like matters - you have to be able totell what they are and which way up theyare. The numbers 6 and 9, and 2 and 5,can easily be confused depending onwhich way up they are and how they’rewritten.

Hand-held devices...but which hand?

People with diabetes use a pen-shapeddevice to give them insulin. These have a

number display built into them. You rotatea dial at the end of the pen to change thedose. Most people are right-handed sohold the pen in their left hand and rotatethe dial with their right. One of the earlyinsulin pens had a number display with a design flaw that meant you couldn’tactually tell which way up the numberswere. It would show the correct dose ifyou were right handed but showed thewrong dose to left-handed people, whowere holding the pen ‘upside down’.Unless you know which way is meant tobe ‘up’ you can’t tell if you’re going togive a dose of 6 or 9 units, or even 12instead of 21? Fortunately the problemwas noticed and because the lesson waslearned, insulin pens are now much safer.

CHI+MED’s Harold Thimbleby fromSwansea University has been worryingabout this kind of thing, and how to makesure numbers on medical devices are lesslikely to cause problems. A common wayfor gadgets to display numbers is to use a‘7 segment display’. The number 8 usesall 7 segments, made up of vertical andhorizontal bars. By either showing orhiding other segments you can make up

all the numbers from 0 to 9. They can bepart of the problem though because manydigits on a 7 segment display areidentical to others when flipped.

Harold has pointed out that while 7segment displays are simple and great forsome things they’re terrible for machineswhere safety is critical. It might not be amatter of life or death if you don’t know ifyour clock is saying or (turn themagazine upside down to see) but gettingnumbers the wrong way round inmedicine is never a good idea. Technologythat turns the world upside downsometimes needs to be used with care.

In an episode of Jonathan Creek ('Themother redcap') the hero realises thata witness told the police the wrongtime. She thought she saw 5:10 on theclock when disturbed in the middle ofthe night, but a glass of water in frontof the hour digit reversed the numberfrom 5 to 2, so the real time was 2:10.

Turning the world

upside down

We’ve all got used to using mobiles: whether tablets, phones or music players. Mobile computers are changing the way we do just about everything. Now hospitals are in on the act. But sometimesturning the world upside down can be a problem, especially if you are left-handed.

Page 20: Machines Making Medicine Safer - WordPress.com › 2014 › ... · 2014-03-22 · Machines Making Medicine Safer It's rare for software problems in medical devices to harm people,

Health and wellbeing arenear the top of everyone’sto do list, even if some ofus don’t actually turn it intoaction. Researchers aroundthe world are exploring howtechnology can help us bemore sporty and healthy.After all, prevention isbetter than cure. So let’srun through a few gadgetsaiming to keep you healthy.

Steps will have to be taken

Accelerometers, tiny devices that measureforce and movement, are common insmartphones and wearable computing. The Fitbit and Nike+ devices, for example,can detect how many steps you take andhow many stairs you climb. You can setyour daily targets or just measure yourgeneral activity and share the results withfriends. You can even share them with theworld in general, so keeping fit becomes a global game. For those less physical,accelerometers are also used in special‘sleep phase’ alarm clocks. They monitorbody movement and wake you up whenyou’re in light sleep mode. That makeswaking up easier and more pleasant.

And the winner is – stairs… eventually

Walk virtuallyanywhere

Omni aim to take game playing a stepfurther. They are developing a platform thatyou stand on and can walk on. As you walk,wearing special shoes, the platform floordetects your footfall. That movement istranslated into movement of your characteron screen. The system will replace a normalgames controller. Instead of pressingbuttons, it’s your actual body walking and running that does the controlling.

Back (page) in the running

www.cs4fn.org

Editors: Paul Curzon, Jo Brodie, Chrystie Myketiak and Peter McOwan, 2014EPSRC supported this issue through CHI+MED which consists of UCL, Swansea, Queen Mary University of London and City Universities. Google support cs4fn.

As computer games increase in theirrealism this could become the new (iftiring) way to get around that virtual town.

And the winner is – World of Warcraft taxi companies

Bike snatcher catcher

Cycling is a popular way to get fit and helpthe environment. An American start-upcompany have created Bike+, a gadget foryour bike. It logs the distances you havepeddled and the routes taken, so you canuse this data to get fitter. It also has anextra twist; one of the things about having a smart bike is that it can be a target forthieves. Bike+ also acts as an alarm, so ifthe bike is moved when it shouldn’t be itsets off an alarm and sends you a text. Youcan track where the thieves have taken your

beloved bike, and hopefully recover itsafely.

And the winner is – the long arm of thelaw and a good pair of legs

Gear of good cheer

A good run can relieve stress, butresearchers at Microsoft have come upwith a new way to monitor how stressedyou are: the smart bra. It measures theheartbeat, skin moisture and activity of the person wearing it, and in testswas able to predict stress levels prettyaccurately. Microsoft are now looking todevelop something more unisex, in theform of a bracelet. Someday soon youclothes or fashion accessories might betelling you to take a run.

And the winner is – pushy clothes, take a hike