Summary Hospital-Level Mortality Indicator (SHMI)

26 November 2013

Overview and background

Summary Hospital-level Mortality Indicator (SHMI) reports on mortality at trust level across the NHS in England

Covers all deaths reported of patients admitted to non-specialist acute NHS trusts who either die while in hospital or within 30 days of discharge

Indicates whether a trust’s mortality ratio is as expected, higher than expected or lower than expected

Produced and published as an experimental official statistic on a quarterly basis by the HSCIC, with the first publication in October 2011

Overview and background

Data sets used

• Hospital Episode Statistics (HES) can be used to identify whether a patient dies in hospital

• HES does not capture deaths occurring outside hospital

• Linking HES to ONS deaths data creates a richer dataset

• Allows the identification of deaths which occur outside of hospital within 30 days of discharge

Exclusion criteria

• Specialist hospitals• Mental health trusts• Community trusts• Day cases• Regular attenders – day

and night• Stillbirths

Contextual indicators

• Also publish a range of contextual indicators alongside the SHMI to aid in its interpretation.

• The following contextual indicators are currently available:

• Palliative care• Admission method• In and out of hospital

deaths• Social deprivation

How SHMI can / cannot be used

What it is intended to be

Indication

Used with other more detailed

indicators

Smoke alarm/ trigger for further

investigation

What it is NOT intended to be

League table

Direct measure/ comparison

A definitive judgement

Why are we producing SHMI?

National review of hospital summary

mortality ratios (HSMR)

Review commissioned because of concerns about• different indicators in use, • the lack of consistency and• lack of clarity about the way some

were being calculated

Review looked at both technical &

audience/use issues Following the recommendations from this review, the Department

of Health committed to implementing the SHMI as the single mortality indicator which

could be adopted across the NHS

How the SHMI was developed

A steering group to define the high-level requirements

A detailed independent statistical modeling and analysis exercise carried out by ScHARR, University of Sheffield

An expert technical group to agree on the specifications

HSCIC commissioned to lead the continued development and improvement of the SHMI, working with a range of stakeholders as well as publishing on a quarterly basis

Quarterly meetings of technical group who act as expert peer reviewers, and review through the Indicator

Assurance Process

Calculation of SHMI

Observed Deaths

• the number of patients who die following treatment at the trust

Expected Deaths

Risk adjusted based on patient

characteristics

• Diagnosis group• Age• Gender• Comorbidities• Admission method

• the number of patients who would be expected to die on the basis of average England figures, given the characteristics of the patients treated there

• calculated using logistic regression

SHMI=ObservedDeathsExpectedDeaths

Calculation of risk

Risk(Age)Risk(Adm

Method)Risk(Sex)

Risk(Co-

morbidity)

logodds

Patient1 Risk

F(log odds)

𝑒log 𝑜𝑑𝑑𝑠

1+𝑒log 𝑜𝑑𝑑𝑠

Calculation of expected deaths

Patient1

Risk

Patient2

Risk

Patient..

Risk

Expected Deaths

Calculation of observed deaths

HESIn

hospital deaths

HES ONSOut of

hospital deaths

Observed Deaths

SHMI funnel plot

• Baseline SHMI value is 1 (observed = expected)

• Providers who do not conform to the national baseline (with associated control limits) are indicating special cause variation

• This variation has not been explained by the baseline model, many possibilities for reasons why, but this warrants a follow-up

Higher than expected

As expected

Lower than expected

Upper Control Limits

Lower Control Limits

SHMI VLAD chart

Variable Life-Adjusted Display (VLAD) charts are for:• a single trust• a single diagnosis group

For each patient observed outcome (0 for survived and 1 for died) is subtracted from the risk of dying and this is plotted cumulatively

Time

A downward trend indicates a run of more deaths than expected

An upward trend indicates a run of fewer deaths than expected

Comparison of control charts

Funnel Plot

• Snapshot of calculated outcome over reporting period

• Identification of outliers• Comparison of overall SHMI value

for trust with national baseline

VLAD Chart

• Cumulative display of longitudinal data over time

• Detect changes in a series of outcomes

• At diagnosis group level

Investigating alerts from control chartsProcess of care

Structure / resource

Patient case-mix

Data

Pyramid of investigation

Lilford R., Mohammed M. A., Spiegelhalter D., Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004; 363: 1147-54.

New developments

Report identifying ‘repeat outliers’ released in January 2013

• Quarterly reporting of ‘repeat outliers’• Taken forward by Sir Bruce Keogh as trusts to be investigated for part of the review into the quality of care

and treatment provided by the NHS, following the Francis Inquiry 2013

Establishing the SHMI as a National Statistic

• Recommended by the Francis Inquiry 2013 (recommendation no. 271)

SHMI Data extract service

• Sharing of underlying record-level SHMI data with trusts• Also providing trusts with VLAD charts for some SHMI diagnosis groups

Contextual indicators

• Development of additional contextual indicators on depth of coding, data quality and transfers between trusts

Issues handling

Issues Log

Methodology Changes

Publication

Current methodological challenges

• Non-specialist trusts with hospices within their organisation

• Guidance on coding for palliative care is subject to interpretation and varies considerably between trusts

Palliative care

• Issues with model convergence for some SHMI diagnosis groups

• Investigating ways of improving this including using more data to build the models, automated re-categorisation to ensure that all case-mix categories have more than a threshold number of events

Model convergence

• SHMI uses Charlson comorbidity index• Carrying out research into using the Elixhauser index,

which includes more conditions• Investigating calibrating the weights of the

comorbidity index on SHMI data

Measures of comorbidity

Questions

Any Questions?

Thank you

Clinical Indicators TeamHealth and Social Care Information Centre

clinical.indicators@hscic.gov.uk

Connect with us

www.hscic.gov.uk@hscicwww.slideshare.net/hscic

0300 303 5678