Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado...
-
Upload
edmund-allen -
Category
Documents
-
view
213 -
download
0
Transcript of Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado...
![Page 1: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/1.jpg)
Monitoring the performance of cardiac surgeonsGraeme L Hickey1; Stuart W Grant2; Camila Caiado3; Iain Buchan1; Ben Bridgewater1,2
1Northwest Institute of BioHealth Informatics, Manchester University2Department of Cardiothoracic Surgery, University Hospital of South Manchester3Department of Mathematical Sciences, Durham University
![Page 2: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/2.jpg)
Background
• Around 35,000 adult cardiac surgery procedures performed each year in UK
• Mortality rate in 2010-11 was 3.4%• Monitoring primarily focuses on in-hospital
mortality
![Page 3: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/3.jpg)
Cardiac surgery performance in the spotlight
BMJ 2005; 330 doi: 10.1136/bmj.330.7490.506 (Published 3 March 2005)Cite this as: BMJ 2005;330:506
![Page 4: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/4.jpg)
Today’s monitoring programme
• National and local-level audits • Revalidation• Dr. Foster Health• FOI requests
![Page 5: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/5.jpg)
The framework
Cardiac surgery Input data locally Uploaded periodically to central database
Aim: 3 monthsReality: 1 year
Statistician + cliniciansCQC website National audit
Aim: <1 yearReality: 3 years
![Page 6: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/6.jpg)
Risk adjustment• Not all surgeons do the same caseload
• NHS surgeon• Does emergency surgery• Specialism in mitral valve
repairs
• Does a lot of private surgery
• Mostly routine elective isolated bypass
Crude mortality = 6.1% Crude mortality = 1.4%
Risk-adjusted mortality = 3.1% Risk-adjusted mortality = 2.9%
Need to risk adjust!
![Page 7: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/7.jpg)
Modelling
• Model for 1999-2012: EuroSCORE– Logistic regression– estimated using prospectively collected during
1995– In-hospital mortality (binary) outcome– Adjustment for 18 risk factors
• Replaced in 2012 by a contemporary model: EuroSCORE II
![Page 8: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/8.jpg)
Monitoring methodology
• Funnel plot– Fixed summary over 3 years of data– Helps identify ‘outlier’ units– Doesn’t address whether hospitals are getting
worse• Variable life adjusted display (VLAD) plot– Intuitive dynamic summary– Doesn’t identify when a unit is an outlier
![Page 9: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/9.jpg)
Funnel plot
Number of cardiac procedures
Risk
-adj
uste
d m
orta
lity
prop
ortio
nAll elective & urgent cardiac surgery in England & Wales
warrants closer investigation
![Page 10: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/10.jpg)
VLAD plot
The tipping point
![Page 11: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/11.jpg)
Outlier ≠ poor performance
• An ‘outlier’ surgeon or hospital (good or bad) might be explainable
• Missing data• Input software
errors• Registry cleaning
errors
![Page 12: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/12.jpg)
Question: What’s wrong with this?All elective & urgent cardiac surgery in England & Wales
Number of cardiac procedures
Risk
-adj
uste
d m
orta
lity
prop
ortio
n
![Page 13: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/13.jpg)
Answer: Model over predicts mortality
Predicted mortality 2.7x greater than observed mortality!
![Page 14: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/14.jpg)
• Observed mortality is decreasing– better surgical tools– improvements in post-
surgery treatment
• Predicted mortality is increasing– increase in older
patients– more complex
procedures
Dynamics of cardiac surgeryMortality proportion
![Page 15: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/15.jpg)
Dynamics of risk
North-South divide?
![Page 16: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/16.jpg)
Patching the problem: short-term
• Update the standard model– recalibration– model re-fit– shrinkage
• Adopt new model– requires validation
• Subgroup analyses– should some procedures be excluded from monitoring?– a statistical + political grey-zone
![Page 17: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/17.jpg)
Patching the problem: long-term
• Better modelling approaches• Online ‘live’ audit reporting + database
querying
![Page 18: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/18.jpg)
Dynamical modelling
• Problem: need a model that stays contemporary
• Some possible solutions:– do nothing– develop a new model every x-years– a moving window recalibration or re-fit – dynamic generalized linear modelling (DGLM)
![Page 19: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/19.jpg)
DGLMM
odel
coe
ffici
ents
(log
-odd
s)
Re-fit Piecewise re-fit
![Page 20: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/20.jpg)
DGLM
• The largest change is coming from intercept
• Latent variables affecting risk?
• Doing nothing is not an option
• Acting periodically might be to late
Intercept
Re-fitPiecewise re-fit
Coeffi
cien
t
![Page 21: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/21.jpg)
Online toolsN
ation
al A
udits
Mon
itorin
g To
ols
Dat
abas
e Q
uery
ing
e-lab
![Page 22: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/22.jpg)
Summary
• Current infrastructure incompatible with developments in healthcare monitoring & surveillance
• Real-world data is not perfect which increases uncertainty + potentially a source of bias
• Modelling for contemporary cardiac surgery needs to adapt to changing dynamics
![Page 23: Monitoring the performance of cardiac surgeons Graeme L Hickey 1 ; Stuart W Grant 2 ; Camila Caiado 3 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f0e5503460f94c22939/html5/thumbnails/23.jpg)
This research was partly funded by Heart Research UK (Grant number RG2583)
Any questions?