Is healthcare getting safer?The challenge of measurement

Charles Vincent Department of Psychology &

Oxford Academic Health Science Network

Evangelists & snails

‘Run don’t walk’

‘The correct question is whether there is a rationale for withholding critical care resources from critically ill patients outside the intensive care unit. The answer is obvious. No’

Walk, don’t run

‘In view of the limitations of the evidence and the heterogeneity of study results it seems premature to declare Rapid Response Teams as the standard of care’.

Davidoff, 2011

Table 4.3 Adverse events in acute hospitals in ten countries

Cutting error and harm by 50% within 5 years

UK National Reporting & Learning System

Hospital Episode Statistics: 11.8M hospital admissions in England 2004/5

But incident reporting only detects 5% of

harmful events

Safety interventions The challenge of scaling up

Cumulative incidence radiologically confirmed thrombosis

Kreckler et al, 2010

Intensive careOperating theatre

• Major complication rate decreased 36%

• Mortality decreased 47%

• Post-op infection decreased 48%

• Central line infection rates decreased 66%

• Quarterly infection rate in most ICU’s <1%

• Estimated saving of $175 million

• Potentially more than 1500 lives saved

Major successes in focal clinical areas

Team training and surgical mortality

After controlling for baseline differences the 74 trained

facilities experienced significant decrease in

mortality of 18% as compared with 7% in the non trained

facilities

Neilly et al, JAMA 2010

Safer Patients InitiativeTo reduce adverse events by 50% in 24 hospitals

11

SPI programme elements 4SPI programme elements 4

Improvement = shifting the level of the process in the desirable direction (A) or

reducing variation (B)

A

BA

BA

BPoint of initial intervention

Improvement = shifting the level of the process in the desirable direction (A) or

reducing variation (B)

A

BA

BA

BPoint of initial intervention

Annotated Run Charts

80 metrics (34 standard)

Process analysis

10

SPI programme elements 3SPI programme elements 3

Incremental spread

Iterative development of

local innovations

9

SPI programme elements 2SPI programme elements 2

8

SPI programme elements 1SPI programme elements 1

8

Breakthrough Series Model

Programme model

Change elements Process measurement

QI methodology

Safer Patients InitiativeParticipating hospital site

Collaborative learning

Expert support

SPI programme elements: Change package

Commentaries on patient safety in the United States

five years after the publication of to key reports

on patient safety in 2000 were characterised by some despair

at an apparent lack of progress. Our data suggest

that a more encouraging story on patient safety in the NHS

can now be told

Benning et al, 2011

The Achievements of SPI

Inspirational and important legacy Objectives over ambitious Organisations in different states of readiness First major UK safety initiative that took

evaluation seriously Simply getting basic clinical data and measures

was a major challenge

Assessing safety interventions at population level

We do not know whether we are making progress or not

Temporal trends in rates of patient harm: |United States

Landrigan et al, NEJM 2011

Making Care Safer. Preventable hospital-acquired conditions would decrease by 40% compared to 2010.

This would mean 1.8 million fewer injuries to patients.

Improving Care Transitions. Preventable complications during a transition be decreased so that hospital readmissions would be reduced by 20% compared to 2010.

This would mean 1.6 million patients recovering without suffering a preventable complication requiring re-hospitalization.

Did Hospital Engagement Networks Actually Improve Care?

‘Weak study design and methods, combined with a lack of transparency and rigour in evaluation …’

‘These numbers appear impressive but given the publicly available data and the approach CMS used it’s nearly impossible to tell whether the PPP actually led to better care’

(Pronovost & Jha, NEJM 2014)

Focus on improvement Menu of national priorities Capacity and capability

building Measurement Leadership Evaluation

15 Patient Safety CollaborativesHosted by Academic Health

Science Networks

Aspiration and realism. The pace of change?

Measurement & Evaluation in Patient Safety Programmes

Our major challenge will be to demonstrate change (rather than activity)

This has bedevilled all safety programmes in NHS Measurement is therefore our number 1 priority because:

– It focuses minds and priorities– It has been the major headache for all safety programmes– The time taken to get measures in place has been consistently

underestimated– It is essential for the programme teams to function effectively– It is fundamental to evaluation

We owe this to patients and carers

Information should include the perspective of patients and their families; measures of harm;

measures of the reliability of critical safety processes; on practices that encourage the monitoring of safety;

on the capacity to anticipate safety problems; on the capacity to respond and learn from safety information.