DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed...
-
Upload
imogen-anderson -
Category
Documents
-
view
212 -
download
0
Transcript of DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed...
![Page 1: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/1.jpg)
DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY
Towards National Definitions and
Agreed StandardsDr Peter Jones MSc EBHC (Oxon) FACEM
With thanks to Dr Alana Harper FACEM and Dr James LeFevre FACEM
MOH Forum, Wellington 5/5/2014
![Page 2: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/2.jpg)
Why Bother?
Each ED could just decide what to measure and how to measure it
Problems Time Resources Skills Duplication (x26) Accuracy Comparisons
‘Admission’=3hrs?
To maximise the return for effort in a resource constrained environment
![Page 3: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/3.jpg)
Aim of this presentation
How to operationalise the MOH suite of quality measures using a real world example Data definitions / development of a ‘Data Dictionary’ Data collection for process and clinical outcomes
![Page 4: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/4.jpg)
My QI Background Auckland City Hospital / ADHB
Morbidity / Mortality 2000-05 Time to Thrombolysis KPI 2000-05 ED Ultrasound Credentialing 2000-03 Procedures Database 2000-05 AHQAS 2001 Cardiac Arrest Documentation 2000-2010
UHCW NHS Trust Audit Lead 2005-06
ACEM Quality Management Subcommittee 2011-current
SOPH PhD Student: Best measure of ED Overcrowding?
![Page 5: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/5.jpg)
Shorter Stays in ED National Research Project
ADHB/ SOPH Auckland University 2009-current Health Policy, Effective Practice, Epidemiology,
Māori Health, Health Economics, Biostatistics, Clinicians
HRC funded 10-588 MREC approved MEC 10/06/060 Multi-stream Mixed-Methods Research
What was done to implement the SSED target? What effect on markers of care? Lessons for future health/public service policy
Kaupapa Māori Research Approach
![Page 6: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/6.jpg)
SSED NRP Stream 2 Quality of Care
Quantitative analysis 13 ‘Quality’ Measures
Process and Outcome Nationwide (routine flow data) 4 Case Study Sites (clinical markers)
Richness of information Target results Q1 2010
Any difference 2006-08 vs 2010-12? Adjust for Ethnicity / Age / Deprivation
![Page 7: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/7.jpg)
SSED NRP Quality
Outcomes Primary (Nationwide)
ED LOS Access Block (wait for admission >8hr in ED) Overcrowding
Hospital LOS Re-attendance rates within 48 hours of discharge Re-admission rates within 28 days of discharge
![Page 8: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/8.jpg)
SSED NRP Quality Outcomes
Secondary Mortality (N)
hospital inpatients for ED attenders at 10, 30 and 90 days
Time to treatment in acute asthma (4 sites) Time to reperfusion for myocardial infarction (1 site) Time to theatre for fractured neck of femur (1 site) Time to appendectomy for acute appendicitis (1 site) Time to antibiotics for severe infections (1 site) Proportion of patients who leave without being seen
(N) ‘Gaming’ the target (N)
a spike of ED discharges at or near the target time (N) digit bias in recording time of ED discharge (N) re-designation of ED patients to ‘stop the clock’ (CS ?N)
![Page 9: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/9.jpg)
SSED NRP Quality Clinical Markers
Clinical Quality Markers Selection Literature review / Evidence Search Reference Group Meeting December 2010
![Page 10: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/10.jpg)
SSED NRP Quality Clinical Markers
Clinical Quality Markers Selection Critical Appraisal of Quality Indicators (QICA)
![Page 11: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/11.jpg)
SSED NRP Quality Data Required
Two Sources, three types of Information NZHIS
Clinical diagnosis data from 4 case sites (ICD) DHBs
Routinely collected process data
ED and acute direct inpatient admissions (PIMS) Hospital Bed Occupancy
Census at night (Bed Management System)
![Page 12: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/12.jpg)
SSED NRP QualityData Collection Plan
7yrs Presentations
Site SpecificClinical Indicators
Process Indicators
![Page 13: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/13.jpg)
SSED NRP Quality Data Dictionary
![Page 14: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/14.jpg)
![Page 15: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/15.jpg)
![Page 16: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/16.jpg)
![Page 17: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/17.jpg)
SSED QualityData Dictionary
Data elements piloted in consultation with analysts from three DHBs
Different names for same process e.g. Episode=Event=Visit=Case
PIMS different
And NZHIS No time stamped process data Highlighted inconsistent submission from DHBs
![Page 18: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/18.jpg)
SSED NRP QualityData Dictionary
Then consulted all 20 DHBs Different PIMS collected data in different ways Not all data elements currently defined Some ED & Inpatient systems not integrated
Level of data capture differs Capture patient event separately Not all Process measures captured by all DHBs
![Page 19: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/19.jpg)
SSED NRPData Dictionary
Dictionary Facilitated Data Cleansing Convert different DHB data format to unified set Identify outliers and validate Link NZHIS data with DHB data
NNPAC/NMDS Identify duplicates from DHB data DHB data has elements missing from NNPAC/NMDS
![Page 20: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/20.jpg)
Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
2006 2007 2008 2009 2010 2011
4
5
6
7
8
9
10
11
12
0
100
200
300
400
500
600
700
800
900
1000
Mean ED LOSData Quality versus true outliers
Mean ED LOS (Hours)
Count of ED LOS > 24 Hours
TIme
LO
S (
Ho
urs
)
Nu
mb
er o
f P
atie
nts
Per
Mo
nth
![Page 21: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/21.jpg)
Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
2006 2007 2008 2009 2010 2011
4
5
6
7
8
9
10
11
12
0
100
200
300
400
500
600
700
800
900
1000
Mean ED LOSData Quality versus true outliers
Mean ED LOS Outliers Fixed
Count of ED LOS > 24 Hours
TIme
LO
S (
Ho
urs
)
Nu
mb
er o
f P
atie
nts
Per
Mo
nth
![Page 22: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/22.jpg)
SSED Indicator Selection and Data Dictionary Resources Required 24 months work 0.5 FTE Emergency Medicine Specialist 1 FTE Research Fellow 1FTE Data Manager Office / Dedicated desk space PCs (high spec / dual screen) Software (reference management / pdf writer / PIMS) Online Journal Access Monthly team meetings Academic and Administrative support Liaison with NZHIS and all DHB IS departments
![Page 23: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/23.jpg)
SSED NRP Clinical QualityData Collection
Site SpecificClinical Indicators
• List of ICD codes to NZHISJ45 (0,1,8,9); J46
• Events with that ICD code in each time period for each site
Date / NHI / Demographics
• Random sample events
• List of NHIs to the sites• Manual collection of data
Trained senior clinician data collectorsmultiple site visits
• Data accuracy / cleaning
![Page 24: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/24.jpg)
SSED NRP Clinical QualityData Collection Tools
‘Intelligent’ spreadsheets Data validation checks / protected formulas
![Page 25: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/25.jpg)
![Page 26: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/26.jpg)
![Page 27: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/27.jpg)
SSED NRP Clinical Quality Data Cleansing
Data Accuracy ≈15% independently checked
Eligibility 283/297 = 97% Primary outcome 366/387 = 95% Comorbidities 1198/1288 = 93%
Data Cleaned Errors corrected where possible No imputation for missing data
Time Stamp Data Also Required Cleaning
![Page 28: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/28.jpg)
SSED Data Collection Resources Required
12 months work 2-3 sets of records per hour for clinical quality indicators
0.5 FTE Emergency Medicine Specialist 1 FTE Research Fellow 1FTE Data Manager Office / Dedicated desk space PCs (high spec / dual screen) Software (data collection tools) Clinical records departments
Desk / PC / Laptop
![Page 29: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/29.jpg)
Development of Quality Measures and Data Dictionary: What’s Needed? MOH
Data definitions / dictionary IT at each site standardised and audited Standardised ‘intelligent’ Tools for data collection
Web vs local
DHB Dedicated resources for quality
Space / Time / Admin / Clinical Records People (thank you NZMC)
More than lip-service!
![Page 30: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/30.jpg)
Development of Quality Measures and Data Dictionary
Towards National Definitions and Agreed Standards
Questions / Discussion
![Page 31: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/31.jpg)
SSED NRP Stream 2 Quality Method: Clinical Markers
Eight conditions identifiedRepresent whole systemPilot 50 sets notesElectronic DEF with logic (reduce errors)Clinically important difference aprioriSample size 90% power, alpha 0.05 (2 tailed)
![Page 32: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/32.jpg)
SSED NRP Stream 2 Quality Method: Clinical Markers
Sample Size Calculations Not all outcomes could be measured at all sites
10000 sets of notes = not feasible Clinical records departments / our resources
Quality Indicator Critical Appraisal Tool 2 Authors independently appraised the indicators Outcome brought to whole team Different views Compromise
Desire to measure >1 outcome and reflect whole system
1 outcome all sites; 1 other outcome each site
![Page 33: DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY Towards National Definitions and Agreed Standards Dr Peter Jones MSc EBHC (Oxon) FACEM With thanks.](https://reader038.fdocuments.in/reader038/viewer/2022110321/56649cfa5503460f949cc1bf/html5/thumbnails/33.jpg)
SSED NRP Stream 2 Quality Method: Clinical Markers Asthma (30 min difference time to steroid)
Common / relevant / important All ages / Mäori / Pacific Data available / accessible / sample size manageable all sites
Acute Myocardial Infarction (15 min difference time to lysis) Evidence strong but practice changed over time:1 site
Sepsis (60 min difference time to steroid) Evidence moderate / important / relevant / all ages / Mäori / Pacific Sample size issues + difficult data collection: 1 site
Appendectomy (12hr difference time to theatre) Whole system (balance)
Fracture NOF (6hr difference time to theatre) Whole system (balance)