Session 1: Responding to Unwarranted Clinical Variation: A Case

Jean-Frederic Levesque

Chief Executive, Bureau of Health Information

Jean-Frederic Levesque, MD, PhDChief Executive

Session 1: Responding to Unwarranted

Clinical Variation – A Case Study

Measuring Stroke Mortality Variation: What We Learned19 June 2014

NSW Health Symposium

December 5th release: 30-day mortality• Insights into Care report

• Acute myocardial infarction, ischaemic stroke, haemorrhagic stroke, pneumonia, hip fracture surgery

• NSW results and variation within the state

• Hospital profiles

• Up to 21 pages of content for each hospital

• Spotlight on measurement

• Discussion of the approach and sensitivity analyses

• Technical supplement

Why are we reporting this measure?• Mortality following hospitalisation is reported internationally as part of

performance assessments.

• Mortality ratios provide a piece of the picture and are complementary to other quality and safety measures.

• Mortality data is influenced by the performance of local systems, not just hospitals, and of multidisciplinary teams.

• RSMRs are screening tools that provide an indication of where further assessment may be needed.

• Public reporting of mortality results can catalyse improvements in comprehensiveness and appropriateness of care for patients.

What is this measure about?• It compares the ‘observed’ number of deaths in the 30-days

following admission with the number of ‘expected’ deaths.

• Deaths occurring in-hospital and after discharge are counted using linked data sets.

• A statistical model is used to calculate the ‘expected’ number of deaths based on the age, sex and comorbidities of patients.

• Cases are attributed to their first presenting hospital during an episode of care.

• The findings are not appropriate for comparing or ranking hospitals or for identifying avoidable deaths.

Overview of the results

Overview of the results

Funnel plot Ischaemic Stroke

(∆) Patients are assigned to the first admitting hospital in their last period of care. Data for hospitals with an expected mortality of < 1 are supressed.

(μ) Hospitals with < 50 patients are not reported publicly. Deaths are from all causes, in or out of hospital. Data exclude AMI STEMI-not specified (ICD-10-AM I21.9).

Source: SAPHaRI, Centre for Epidemiology and Evidence NSW Ministry of Health.

Ischaemic stroke 30-day risk-standardised mortality ratio, NSW public hospitals, July 2009 – June 2012 ∆ μ

Analysis by peer group Ischaemic strokeIschaemic stroke 30-day risk-standardised mortality ratio, by peer group July 2009 – June 2012

Hospital profile: Summary dashboard

Hospital profile: Stability of results

Following the report’s release

• Various hospitals have considered the results and identified where improvements could be made.

• Results were reviewed alongside other quality and safety measures, such as clinical audit and review panels.

• Eight clinical settings contacted us to obtain clarifications on the measures or more detailed information.

• Future provision of updates and complementary analyses will be provided.

What have we learned?• Mortality is an easily understood outcomes that requires very

sophisticated methods to be reported in a fair way

• Understanding contextual and system-level factors is important in developing mortality measures (e.g. impact of transfers)

• Partnering with clinical leaders supports the development of clinically-relevant measures and the rigorous validation

• Internal and public reporting of information acts in synergy to raise awareness and catalyse assessments

• Outcomes that are sensitive but crucial for patients can be part of the range of measures reported publicly to support accountability and meaningful discussion

Acknowledgements• Kim Sutherland, Director, System and Thematic Reports,

Bureau of Health Information

• Doug Lincoln, Lead Analyst, Bureau of Health Information

• Sadaf Marashi-Poor, Senior Analyst, Bureau of Health Information

• Kerrin Bleicher, Analyst, Bureau of Health Information

• Sally Prisk, Graphic Designer, Bureau of Health Information

• All BHI staff