Does pay for performance save lives? Martin Roland University of Cambridge UK.
-
Upload
elvis-gudger -
Category
Documents
-
view
214 -
download
0
Transcript of Does pay for performance save lives? Martin Roland University of Cambridge UK.
Does pay for performance save lives?
Martin Roland
University of Cambridge
UK
Does pay for performance save lives?
University of Manchester
• Matt Sutton, Ruth Boaden, Søren Rud Kristensen, Rachel Meacock, Alex Turner
Warwick Business School
• Ruth MacDonald
University of Cambridge
• Martin Roland
All authors declare that they have no conflict of interest. The study was funded by the National Institute of Health Research
What’s the evidence on pay for performance?
Current literature suggests:
• Clear evidence that pay for performance can improve the process of care
• Not a ‘magic bullet’ – effects likely to be limited in the absence of other quality improvement measures
• Very limited evidence of impact on outcome
• P4P scheme introduced in 2008 in all 24 NHS hospitals
in the North West of England (population 6.8m)
• Process indicators for acute myocardial infarction,
heart failure, pneumonia, hip and knee surgery and
coronary artery bypass surgery
Evaluation of ‘Advancing Quality’
• Months 1-12: Tournament system in which only the
top performers received a bonus (top quartile
received extra 4% second quartile extra 2%)
• Months 13-18: Bonuses for both achieving targets
and improving score
• Months 19-42: Penalties if hospitals failed to achieve
targets
‘Advancing Quality’ payment rules
Impact on mortality for acute myocardial infarction, heart failure and pneumonia in:
• First 18 months (Sutton et al NEJM 2012;367:1821–8)
• Next 24 months (NEJM ‘revise and resubmit’)
We did not include hip and knee surgery (low mortality) or
coronary bypass surgery (not carried out in most hospitals)
Evaluation of ‘Advancing Quality’ over 42 months
Study design
We studied 30 day in-hospital mortality comparing mortality for acute myocardial infarction, heart failure and pneumonia using a difference in difference analysis with three sets of controls:
• Mortality for these conditions in all other 132 hospitals in England
• Mortality for non incentivised conditions in: a) hospitals in the North West of England and b) all other hospitals in England
Selection of non-incentivised conditions
• not clinically linked to any incentivized condition
• sufficient volume (over 9,000 admissions in England per year)
• 30-day mortality over 6%
• more than 80% of deaths within 30 days occurring in a hospital
• Six conditions met these criteria: acute renal failure, alcoholic liver disease, intracranial injury, and intestinal obstruction, pulmonary embolism and duodenal ulcer
Data available for analysis
• Used routine hospital activity data collected from NHS hospitals
• All admissions for 18 months before and 42 months after the introduction of the scheme
• Patients treated at the 24 NHS hospitals in the North West and the 132 NHS hospitals in all other regions of England
Number of admissions in each analysis
18 months 48 months
Pneumonia 410,384 761,954
Heart failure 201,003 338,921
Acute myocardial infarction
245,187 390,652
Non-incentivised conditions
241,009 333,991
Includes admissions for 18 months prior to P4P scheme
Difference in differences analyses controlled for:
• Gender
• Age
• Type of admission (planned/unplanned, transfer)
• Location from which the patient was admitted
• 31 co-morbidities (Elixhauser) derived from secondary diagnostic codes
Incentivised conditions
Incentivised conditions
Non -incentivised conditions
Comparison 2
Incentivised conditions
Non -incentivised conditions
Comparison 3
Incentivised conditions
Comparison 1Incentivised conditions
24 hospitals in North West England
132 hospitals in the rest of England
Difference-in-differences analyses
Changes in mortality summarised as triple difference in differences analysis
Sensitivity analyses
• Controlling for baseline mortality
• Controlling for changes in patient volumes
• Controlling for hospital quality and financial rating (data from national regulator)
• Controlling for hospital type (teaching)
• Different method of classifying co-morbidities (Charlson )
• Checking pre-intervention trends in mortality no different between incentivised and non-incentivised conditions or between areas.
• Comparing North West with a subset of similar English districts instead of the whole of England
Summary of analysis of first 18 months
• Significant reduction in absolute mortality of 1.3% (relative reduction of 6%)
• Driven by significant reduction for pneumonia and non-significant reductions for MI and heart failure
• Equivalent to 890 fewer deaths in the North West in the 18 month period
Sutton M, Nikilova S, Boaden R, Lester H, McDonald R, Roland M. Reduced mortality with hospital pay for performance in England. New England Journal of Medicine 2012;367:1821–8
Analysis of the next 24 months
• Between-region difference-in-differences analysis
• Time trends for each of the 20 quarter year periods
• Separate estimates of impact of short and long term periods
What happened in the second time period?
• Mortality continued to fall in the North West, but there was no longer a significant difference compared to controls
• Mortality for incentivized conditions fell more in control hospitals than in Advancing Quality hospitals (0.7 percentage points more, 95% CI 0.3 to 1.2)
• Mortality for non-incentivized conditions fell more in Advancing Quality hospitals than in control hospitals (1.2 percentage points more, 95% CI 0.4 to 2.0).
What happened in the second time period?
Incentivised conditions
Non-incentivised conditions
Advancing Quality hospitals (North West)
First 18 months -1.6% -0.5%
Next 24 months -1.8% -2.9%
Rest of England (controls)
First 18 months -0.8% -1.2%
Next 24 months -2.3% -1.7%
Estimates are from weighted least-squares regression models Results are robust to a range of specifications of standard errors and weights
Possible explanations
• Mortality for incentivized conditions fell more in control hospitals than in Advancing Quality hospitals
Did the other hospitals start to use the AQ quality indicators?
Possible explanations
• Mortality for incentivized conditions fell more in control hospitals than in Advancing Quality hospitals
Did the other hospitals start to use the AQ quality indicators?
• Mortality for non-incentivized conditions fell more in Advancing Quality hospitals than in control hospitals
Were there ‘spillover effects’ on other aspects of care in the intervention hospitals?
Possible explanations
Did the other hospitals start to use the AQ quality indicators?
• The results of the AQ pilot were widely disseminated and given national publicity
• Two other regions of the country developed P4P schemes using the AQ indicators
• The reduction in mortality in these regions was more than in other regions of England
Possible explanations
Were there ‘spillover effects’ on other aspects of care in the intervention hospitals?
• We looked at which doctors cared for patients with the non-incentivised conditions
• There was substantial overlap between doctors caring for the incentivised conditions and two of the non-incentivised ones – acute renal failure and alcoholic liver disease
• The fall in mortality for these two non-incentivised conditions was greater than for the other non-incentivised conditions
Lessons for evaluation
Impact on outcomes may not be long-lasting
Spillover effects may hinder evaluation by masking true effects, e.g.
• into un-incentivised hospitals
• onto un-incentivised conditions
(spillover effects are deliberate)
How did we explain the reduction in mortality?
• Not solely on the basis of changes in process measures
• Wide range of quality improvement strategies
• New data collection systems and feedback on performance
• Employment of new specialist nurses
• Regular face to face learning events between hospitals
Why was it different from the US (HQID)?
• Self-selected hospitals in US
• Larger bonuses
• Greater probability of earning a bonus
• More face to face learning between hospitals
Pay for performance can have a beneficial impact on outcomes
The causal pathway is likely to be multi-factorial and not just related to improvements in incentivised aspects of care
The impact may not be long-lasting, but spillover effects (which are welcome) may mask longer term improvement
Quality indicators in the Advancing Quality P4P scheme
Pneumonia• Blood cultures performed in the ER prior to initial antibiotics • Adult smoking cessation advice/counselling• Initial antibiotic received within six hours of hospital arrival• Initial antibiotic selection in immuno-competent patients• Oxygenation assessment
Heart failure• Discharge instructions• Adult smoking cessation advice/counselling• ACEI or angiotensin receptor blocker for left ventricular systolic dysfunction• Left ventricular systolic function assessment
Acute myocardial infarction• Adult smoking cessation advice/counselling• Beta blocker prescribed at discharge• Aspirin at arrival• Aspirin prescribed at discharge• Fibrinolytic therapy received within 30 minutes of hospital arrival• ACEI or angiotensin receptor blocker for left ventricular systolic dysfunction
Table 1: Average hospital achievement on the incentivized indicators in the North West of England in the short and long-term periods
Condition Indicator name Short Term Long Term Difference
Pneumonia
Blood cultures performed in the emergency department prior to initial antibiotics received in hospital 63.1 81.9 18.7 Adult smoking cessation advice/counselling 44.9 62.7 17.8 Initial antibiotic received within six hours of hospital arrival 66.2 76.2 10.0 Initial antibiotic selection in immunocompetent patients 82.7 91.2 8.5 Oxygenation assessment 97.6 99.5 1.9
Heart Failure
Discharge instructions 29.2 55.1 25.9 Adult smoking cessation advice/counselling 55.6 75.7 20.1 Angiotensin converting enzyme inhibitor or angiotensin receptor blocker for left ventricular systolic dysfunction 90.1 94.9 4.8 Left ventricular systolic function assessment 89.3 93.6 4.3
Acute Myocardial Infarction
Adult smoking cessation advice/counselling 86.8 94.1 7.3 Beta blocker prescribed at discharge 93.8 97.6 3.7 Aspirin at arrival 97.1 98.8 1.8 Aspirin prescribed at discharge 98.3 99.3 1.1 Fibrinolytic therapy received within 30 minutes of hospital arrival 84.4 85.3 0.9 Angiotensin converting enzyme inhibitor or angiotensin receptor blocker for left ventricular systolic dysfunction 97.5 98.0 0.5
Note: The table shows the percentage of patients for whom an indicator was met on the incentivized indicators in the North West of England. The short term period covers the first 18 months of the program. The long term period includes months 19-42 of the program. Within each condition the measures are ranked in descending order of the magnitude of the increase in achievement between the short- and long-term periods.