Reliability Theory and its Application to Healthcare

Aims of session

• Introduction to reliability theory – the framework and the three step model

• Highly reliable organisations – who are they? Can we learn from them?

• Healthcare as a highly reliable industry – designing reliable systems of care

• Care bundles – a reliability approach

Reliability in healthcare• Healthcare is a high hazard industryHealthcare is a high hazard industry• We are not able to reliably deliver healthcare to We are not able to reliably deliver healthcare to

all of our patients all of the time.all of our patients all of the time.• Approx. 10% (900,000) of patients admitted to

hospital experience an incident.• 72,000 of these incidents/adverse events

contribute to the death of patients• Many go unrecognised

Patient safety – a global issue

0

2

4

6

8

10

12

14

16

18

% of acute admissions

USA 3.7%

Australia 16.6%

England 10.8%

Denmark 9%

New Zealand 12.9%

Canada 7.5%

Japan 11%

Impact

Direct costs:• in England healthcare associated infections are

estimated to cost over £1 billion pounds per year• on average, preventable drug events resulted in

an additional 4.6 days in length of stay• estimated cost of preventable adverse events in

USA is $10.1 billion (Leape et al 1993)

Trust Board Away Day – October 2005

Is medicine a high-reliability industry?• The practice of medicine involves complex systems in

which humans play a key role• Procedures are very technical and sometimes risky• Medicine should be a high-reliability industry• Unfortunately literature shows that it is fraught with error,

can be unsafe, and at times is not effective• The potential for error and system failure is always there• Things happen on a daily basis: staff go off sick,

equipment doesn’t work, people forget to do something - we are all human no matter how diligent

• This is a normal part of a complex healthcare system

What is reliability science?

• Reliability principles are used successfully in industries such as manufacturing and air travel to help evaluate, calculate and improve the overall reliability of complex systems

• These can be used to design systems that compensate for the limits of human ability, can improve safety and the rate at which a system consistently produces the desired outcomes

How is it measured?• Reliability is measured as the inverse of

the systems failure rate• A system that has a defect rate of one in

ten or 10% performs at a level of 10 – 1• Reliability is defined as failure-free

operation over time• Reliability = number of actions that

achieve the intended result, divided by total number of actions taken

A reliability framework• 10 – 1 performance on process measures indicates no

articulated common process and an emphasis on training and reminders (international studies of adverse events in hospitals shows an error rate of 10% suggesting a level at which most organisations currently perform)

• 10 – 2 performance on process measures indicates processes intentionally designed with tools and concepts based on the principles of human factors engineering

• 10 – 3 or better performance on process measures indicates a well designed system with attention to processes structure and their relationship to outcomes

Examples• 10-1= 80 or 90% success, 1 or 2 failures out of 10

opportunities ( A chaotic process)– B-blockers after acute MI

• 10-2 = 5 failures or less out of 100 opportunities– Mortality in general surgery

• 10-3= 5 failures or less out of 1000 opportunities - Mortality in routine anaesthesia

• 10-4 = 5 failures or less out of 10,000 opportunities

A chaotic process is failure in greater than 20% of opportunities

Almost all studies that investigate the reliability of the application of clinical evidence conclude that it is 10-1

Improving reliability

Level I Intent, vigilance & hard work

Level II Design systems for reliability constraints, decision aids, reminders, checklists, bundles

Level III Prevent design for reliability Identify make failures visible Mitigate prevent / treat harm due

to failures

How to reduce variability

• Standardisation Care bundles ICPs Guidelines• Checklists• Improve access to information• Reduce reliance on memory• Constraints• Reduce handovers• Simplify processes

Standardisation concepts

• Standardisation is done to provide the appropriate infrastructure

• The ‘what’ we are standardising based on good medical evidence

• The ‘how’ does not need to be based on good medical evidence but rather on systems knowledge

In a broader context

• Aviation passenger safety is measured at 10-6

• Nuclear power plants must demonstrate a design capable of operating at 10-6 before they can be built

IHI three-tiered strategy for designing reliable care systems

1. Prevent failure

2. Identify and mitigate failure – identify failure when it occurs and intercede before harm is caused, or mitigate the harm caused by failures that are not detected

3. Redesign the process based on the critical failures identified

Designing effective and reliable systems

• Have simple rules – complex systems best handled by this• Feature redundancy – offers multiple layers of defence from error• Incorporate forcing functions – a mechanism that makes it easy to

do the right thing and hard to do the wrong thing (i.e. on a plane the toilet light cannot be turned on without locking the door first)

• Ensure people cannot work around the system first – understand why people develop workarounds

• Minimise reliance on human memory• Allow the expertise of the people performing the work to be used –

standardised protocols provide a systematic approach • Incorporate technology where possible• Communicate the advantages of the system to clinicians – if staff do

not see this they will develop workarounds• Consider what happens if the system fails – be prepared

How Hazardous Is Healthcare?(Leape and Amalberti)

1

10

100

1,000

10,000

100,000

1 10 100 1,000 10,000 100,000 1,000,000

10,000,000

Number of encounters for each fatality

To

tal

liv

es

lo

st

pe

r y

ea

r

REGULATEDHAZARDOUS(>1/1000)

ULTRA-SAFE(<1/100K)

Health Care

Mountain Climbing

Bungee Jumping

Driving

Chemical Manufacturing

Chartered Flights

Scheduled Airlines

European Railroads

Nuclear Power

Highly reliable organisations?

• A definition of a HRO is one that is known to be complex and risky, yet safe and effective

• These organisations acknowledge the complexity of their systems create an environment in which individuals can communicate openly about concerns and design systems that make it difficult for failures to occur

• HROs ask ‘what happens when the system fails?’, not ‘What if the system fails?’

Examples of highly reliable organisations

• Aviation

• Nuclear power plants

• Air traffic control centre

• Nuclear aircraft carriers

Learning from highly reliable organisations

• Other highly technical industries bear a similarity to medicine

• Airline industry - thousands of flights take place every day in varying weather conditions. If a significant error occurred the consequences would be dire

• So why is the error rate in aviation not the subject of public and media interest?

Lessons learned the hard way!

The airline industry

• Aviation industry recognised years ago that human error is an inevitable part of doing business

• The industry chose to address error prevention and safety by improving communication, flattening team hierarchy and implementing fail safe systems

• These actions have made aviation a highly reliable industry

High reliability organisations

• Strong organisational culture of reliability• Continuous learning• Effective and varied patterns of communication• Human resource management practices that

support reliability• Adaptable decision-making dynamics• Managing technology• System and human redundancy

The need to apply a Systems Approach

• Failure is predictable and can be detected• Failure arises out of systematic and

organisational factors – not just erratic behaviour of individuals

• High reliability departments create safety by anticipating and planning for unexpected events and future surprises

Can reliability be applied to healthcare?

• Although healthcare is not currently highly reliable, it has the potential to be

• IHI and others believe that applying reliability principles to healthcare has the potential to reduce defects in care or care processes, increase the consistency with which appropriate care is delivered, and improve patient outcomes

• To move in that direction we must overcome one of the largest barriers – the culture of medicine

There is hope• One bright light in the field of healthcare with regard to

high reliability – anaesthetics• No other medical discipline has come as close• Realisation that the weak link in the process was the

people not the technology (1984 Cooper published his study – review of 329 incidents involving anaesthesia in a Massachusetts Hospital identified that nearly 70% of these incidents related to human error

• They have learned lessons and implemented changes that the rest of the healthcare field are just beginning to acknowledge

• In 1954, one out of every 1,500 patients died as a result of problems with their anaesthetic

• In 2001 that risk has dropped to one in every 250,000

Using care bundles to improve reliability

• Bundles demand ‘all or none’ thinking and measurement

• Bundles facilitate identifying failures• Failures are actively used to redesign the

process• Team work and communication proven to

improve

What are they?

• A series of interventions relating to a treatment or intervention

- ventilator bundle

- central Line bundle

- tracheostomy bundle etc• When implemented together will achieve

significantly better outcomes than when implemented individually (IHI 2005)

Why?

• A way of reducing the gap between research and practice in clinical areas

• Promotes evidence-based change• The bundle of care will have a greater effect on

the positive outcome of the patient than if used in isolation

• Reduces variation from unit to unit or clinician to clinician

Care bundles

Based on reliability principles – all or nothing compliance:

• Plane takes off ok, one engine fails during flight, descends ok, lands ok = 75%

• Plane takes off ok, one engine fails during flight, descends badly, crashes on landing= 25%

• Plane takes off ok, engines ok during flight, descends ok, lands ok = 100%

• Overall flight compliance – 66%

Would you want to travel on this airline?

Evidence

• IHI estimates that it could be possible to achieve an 80% reduction in Surgical Site Infections (of which 3% could be fatal) and a 50% reduction in deaths from Acute Myocardial Infarction

• They also estimate that an average bed sized U.S. hospital could save 18 lives from SSI and 108 lives from AMI each year as a result of implementing care bundles

An example

Level of reliability of all 4 elements of ventilator bundle

< 95% compliance

> 95 % compliance

Reduction of Ventilator Acquired Pneumonia

46 %

59 %

L&D % Daily compliance with Care BundleJuly / Aug /Sept/Oct/Nov/Dec03/Jan04/feb/Mar/Apr04

020406080

100120

1 12 23 3 14 25 5 16 27 8 19 30 10 21 2 13 24 4 15 26 6 17 28 10 21Ap

r 12 23

% Dailycomp

Target

Outcomes

• Evidence that the unit is achieving quality care and doing the right thing for the right patient

• Average length of stay is reducing

• Sedation costs reduced – financial savings

Central line bundle

Central line infection rate

Making the move• Need to move towards a culture focused on safety and

reliability• Leadership driven with staff focused on safe and reliable

care• Adoption of standardised methods of communication and

in the creation of an environment in which people interact collaboratively and feel free to speak up if they see something worrying

• Engineer systems with redundancy and safeguards that make doing the wrong thing difficult

• Create a learning environment in which little problems are seen as indicators of deeper potential faults to be addressed proactively