1.Background Methods Findings Summary The Relationship Between Quality of Care and Choice of Clinical Computing System GP Performance Under the Quality and Outcomes Framework Evan Kontopantelis12 Iain Buchan1 David Reeves12 Kath Checkland12 Tim Doran3 1Institute of Population Health, University of Manchester 2NIHR School for Primary Care Research 3Department of Health Sciences, University of York CHI meeting, 10 July 2013 Kontopantelis Clinical Computing Systems and QOF

2. Background Methods Findings Summary Outline 1 Background 2 Methods 3 Findings Variation Model results 4 Summary Kontopantelis Clinical Computing Systems and QOF 3. Background Methods Findings Summary Clinical Computing Systems Promoted as a means to improve the quality of health care, with advantages over paper based systems: improved data recording integration and accessibility feedback and alerts Drive improvements in efciency, process performance, clinical decision making and medication safety? In practice the results of implementing information technology systems have been mixed: variable quality of software systems by multiple providers clinicians adapting at different rates to systems that may not address their specic requirements or challenge their approach to existing practice Kontopantelis Clinical Computing Systems and QOF 4. Background Methods Findings Summary Clinical Computing Systems in the UK Primary Care Developed from multiple systems in the 1980s into a handful using Read codes by 2000 From 1998 practices partially subsidised for the costs of installing clinical computing systems Full subsidies provided from 2003 in preparation for the implementation of a national pay-for-performance scheme, the Quality and Outcomes Framework, in April 2004 Kontopantelis Clinical Computing Systems and QOF 5. Background Methods Findings Summary The Quality and Outcome Framework (QOF) Provides large nancial bonuses to practices based on achievement on over 100 quality of care indicators Practices awarded points for each quality indicator based on the proportion of patients for whom targets are met Each point worth 126, adjusted for the relative prevalence of the disease and the size of the practice population Practices can exclude (exception report) inappropriate patients from achievement calculations for various reasons: logistical (e.g. registration close to end of nancial year) clinical (e.g. a contraindication to treatment) patient informed dissent Kontopantelis Clinical Computing Systems and QOF 6. Background Methods Findings Summary The Quality and Outcome Framework (QOF) continued Performance data drawn from clinical computing systems and collated on a national database (the Quality Management and Analysis System - QMAS) systems must conform to standard interoperability requirements QMAS scores on each indicator are used for feedback and payment calculations The QOF is in its 9th year and has cost on average 1bn per annum Kontopantelis Clinical Computing Systems and QOF 7. Background Methods Findings Summary QOF impact Substantial impact on use of clinical computing systems Practices are required to keep disease registers and because bonus payments increase with disease prevalence there is an incentive to case-nd QOF business rules specify criteria and permissible Read codes for identifying patients with particular conditions: greater uniformity of code usage even changing diagnostic behaviour Software providers have adapted their systems to facilitate better practice performance on the QOF: incorporating pop-up alerts and management tools QOF-oriented training programmes for practice staff Kontopantelis Clinical Computing Systems and QOF 8. Background Methods Findings Summary The question There is no independent evidence to date on whether practice performance on the QOF and recorded quality of care is associated with the practices choice of clinical computing system We used a unique dataset to assess these relationships in English family practices operating under the national pay-for-performance scheme. Retrospective study of QOF performance by English family practices from 2007-8 to 2010-11, identifying practice predictors, including choice of clinical computing system Kontopantelis Clinical Computing Systems and QOF 9. Background Methods Findings Summary Data Quality Management and Analysis System (QMAS), which holds data for almost all English practices (over 8,200) Clinical computing systems info not publicly reported; obtained relevant dataset from the HSCIC Practice characteristics and the populations they serve obtained from the General Medical Services (GMS) Statistics database, also provided by the HSCIC Area deprivation, as measured by the IMD, from the Communities and Local Government website Urban/non-urban classication, from the ONS website Data were complete for all practices Kontopantelis Clinical Computing Systems and QOF 10. Background Methods Findings Summary Outcomes Practice performance on the 62 QOF clinical quality indicators that were continually incentivised over the study period Three performance measures used: reported achievement (RA), % of eligible patients for whom targets achieved (not including exception reported patients) population achievement (PA), % of eligible patients for whom targets actually achieved (including exception reported patients) % of QOF points scored (PQ), the metric on which remuneration is based Kontopantelis Clinical Computing Systems and QOF 11. Background Methods Findings Summary Indicators some of the 62 Table A-6: Quality and Outcomes Framework indicators included in the main analysis Indicator Min Threshold Max Threshold Points Description Type of indicator AF 4 40 90 10 Diagnosis confirmed by ECG or specialist Measurement AF 3 40 90 15 Treated with anti-coagulation/anti-platelet Treatment ASTHMA 8 40 80 15 Variability/reversibility measured Measurement ASTHMA 3 40 80 6 Smoking status recorded (ages 14-19) Measurement ASTHMA 6 40 70 20 Care reviewed Measurement BP 4 40 90 20 Blood pressure measured Measurement BP 5 40 70 57 Last blood pressure 150/90 mmHg Outcome CANCER 3 40 90 6 Care reviewed Measurement CHD 2 40 90 7 Referred for exercise testing/specialist assessment Measurement CHD 5 40 90 7 Blood pressure measured Measurement CHD 6 40 70 19 Blood pressure 150/90 mmHg Outcome CHD 7 40 90 7 Total cholesterol measured Measurement CHD 8 40 70 17 Total cholesterol 5 mmol/l Outcome CHD 9 40 90 7 Treated with aspirin/anti-platelet/anti-coagulant Treatment CHD 10 40 60 7 Treated with beta blocker Treatment CHD 11 40 80 7 Treated with ACE-inhibitor (history of myocardial infarction) Treatment CHD 12 40 90 7 Influenza vaccination Treatment HF 2 40 90 6 Diagnosis confirmed by echocardiogram/specialist Measurement HF 3 40 80 10 Treated with ACE inhibitor/angiotensin receptor blocker Treatment Kontopantelis Clinical Computing Systems and QOF 12. Background Methods Findings Summary Outcomes continued Across all three outcome measures (RA, PA and PQ) practice composite scores were calculated Overall, across all 62 clinical indicators By three categories of activity: measurement activities, 35 indicators (e.g. measurement of blood pressure for hypertension patients) treatment activities, 11 indicators (e.g. inuenza immunisation to CHD patients) intermediate outcomes, 16 indicators (e.g. HbA1C level control for diabetes patients) Kontopantelis Clinical Computing Systems and QOF 13. Background Methods Findings Summary Statistical Modelling Multilevel mixed effects multiple linear regression models to identify population, practice, and clinical computing system predictors of RA PA and PQ Model 1: overall summary measure of 62 ind as outcome estimate effect of each clinical system on outcome and assess whether there were signicant differences across clinical systems Model 2: indicator group (measurement, treatment, outcome) summary measure as outcome included interaction terms (clinical system by indicator group) to estimate the system effect in each indicator group and assess differences across clinical systems, in each of the indicator groups Kontopantelis Clinical Computing Systems and QOF 14. Background Methods Findings Summary Statistical Modelling Analyses controlled for year, practice list size, local area deprivation, rurality, type of GP contract, % female patients, % patients aged 65+, mean GP age, % female GPs, % UK qualied GPs, % GP providers (partners, single-handed, or shareholders) Linear predictions, and their 95% CIs, were calculated for each clinical system from the regression models These can be described as mean predicted outcome levels for each factor (clinical system and indicator group by clinical system) that are controlled for all model covariates and set to their mean values in the models, thus allowing us a fairer comparison of the systems Kontopantelis Clinical Computing Systems and QOF 15. Background Methods Findings Summary Variation Model results System usage Fifteen clinical computing systems from eight providers Seven systems accounted for 99% of the market Provider Product Year 2007-8 2008-9 2009-10 2010-11 EMIS LV 3811 (46.0%) 3661 (44.5%) 3498 (42.2%) 3284 (39.9%) In Practice Systems Vision 3 1572 (19.0%) 1599 (19.5%) 1564 (18.9%) 1492 (18.1%) TPP ProdSysOneX 697 (8.4%) 851 (10.4%) 1164 (14.0%) 1466 (17.8%) EMIS PCS 1103 (13.3%) 1160 (14.1%) 1259 (15.2%) 1216 (14.8%) iSoft Synergy 541 (6.5%) 487 (5.9%) 444 (5.4%) 401 (4.9%) Microtest Practice Manager 166 (2.0%) 164 (2.0%) 156 (1.9%) 156 (1.9%) iSoft Premiere 248 (3.0%) 212 (2.6%) 169 (2.0%) 132 (1.6%) Other* 145 (1.8%) 86 (1.0%) 31 (0.4%) 93 (1.1%) Total 8283 8220 8285 8240 * Excluded from analyses: EMISWeb, GV (EMIS); HealthyV5, Crosscare (Healthy Software), Seetec GP Enterprise (Seetec), Ganymede, System 6000 (iSoft), Exeter GP System (Protechnic Exeter Ltd) Kontopantelis Clinical Computing Systems and QOF 16. Background Methods Findings Summary Variation Model results System and supplier distribution North East North West London West Midlands Yorkshire & the Humber South West East Midlands East of England South Central South East Coast (85.4,85.7] (85.1,85.4] (84.8,85.1] (84.5,84.8] (84.2,84.5] [83.9,84.2] LV Vision 3 ProdSysOneX PCS Synergy Practice Manager Premiere NOTE: Chart size proportional to number of practices in area Average practice scores by Strategic Health Authority, 201011 Overall population achievement (62 indicators) and GP systems products North East North West London West Midlands Yorkshire & the Humber South West East Midlands East of England South Central South East Coast (90.2,90.5] (89.9,90.2] (89.6,89.9] (89.3,89.6] (89,89.3] (88.7,89] [88.4,88.7] EMIS In Practice Systems TPP Microtest ISoft NOTE: Chart size proportional to number of practices in area Average practice scores by Strategic Health Authority, 201011 Overall reported achievement (62 indicators) and GP systems suppliers Kontopantelis Clinical Computing Systems and QOF 17. Background Methods Findings Summary Variation Model results Performance and points achieved by Primary Care Trust, 2010-11 (87,88] (86,87] (85,86] (84,85] (83,84] (82,83] (81,82] (80,81] [79,80] Average practice scores by Primary Care Trust, 201011 Overall population achievement (62 indicators) (91,92] (90,91] (89,90] (88,89] (87,88] (86,87] [85,86] Average practice scores by Primary Care Trust, 201011 Overall reported achievement (62 indicators) (99,100] (98,99] (97,98] (96,97] (95,96] (94,95] (93,94] (92,93] [91,92] Average practice scores by Primary Care Trust, 201011 Overall % of points achieved (62 indicators) Kontopantelis Clinical Computing Systems and QOF 18. Background Methods Findings Summary Variation Model results Reported achievement by Primary Care Trust, 2010-11 (95,96] (94,95] (93,94] (92,93] (91,92] (90,91] [89,90] Average practice scores by Primary Care Trust, 201011 Overall reported achievement (35 measurement ind) (92,93] (91,92] (90,91] (89,90] (88,89] (87,88] [86,87] Average practice scores by Primary Care Trust, 201011 Overall reported achievement (16 treatment ind) (83,85] (81,83] (79,81] (77,79] (75,77] [73,75] Average practice scores by Primary Care Trust, 201011 Overall reported achievement (11 outcome ind) Kontopantelis Clinical Computing Systems and QOF 19. Background Methods Findings Summary Variation Model results Population achievement by Primary Care Trust, 2010-11 (92,93] (91,92] (90,91] (89,90] (88,89] (87,88] (86,87] (85,86] [84,85] Average practice scores by Primary Care Trust, 201011 Overall population achievement (35 measurement ind) (83,85] (81,83] (79,81] (77,79] [75,77] Average practice scores by Primary Care Trust, 201011 Overall population achievement (16 treatment ind) (78,80] (76,78] (74,76] (72,74] (70,72] (68,70] [66,68] Average practice scores by Primary Care Trust, 201011 Overall population achievement (11 outcome ind) Kontopantelis Clinical Computing Systems and QOF 20. Background Methods Findings Summary Variation Model results Percentage of points achieved by Primary Care Trust, 2010-11 (98,100] (96,98] (94,96] (92,94] [90,92] Average practice scores by Primary Care Trust, 201011 Overall % of points achieved (35 measurement ind) (99,100] (98,99] (97,98] (96,97] (95,96] (94,95] (93,94] (92,93] [91,92] Average practice scores by Primary Care Trust, 201011 Overall % of points achieved (16 treatment ind) (99,100] (98,99] (97,98] (96,97] (95,96] (94,95] [93,94] Average practice scores by Primary Care Trust, 201011 Overall % of points achieved (11 outcome ind) Kontopantelis Clinical Computing Systems and QOF 21. Background Methods Findings Summary Variation Model results More practice variability by Primary Care Trust, 2010-11 (50,60] (40,50] (30,40] (20,30] (10,20] [0,10] By Primary Care Trust, 201011 Average practice location deprivation (9000,10000] (8000,9000] (7000,8000] (6000,7000] (5000,6000] [4000,5000] By Primary Care Trust, 201011 Average practice list size Kontopantelis Clinical Computing Systems and QOF 22. Background Methods Findings Summary Variation Model results Practice characteristics by system 2010-11 LV Vision 3 ProdSys- OneX PCS Synergy Practice Manager Premiere List size 6903 (4174) 6681 (4286) 6587 (4333) 5715 (3922) 8053 (4287) 6654 (3791) 7104 (3703) IMD 2010 25.2 (17.0) 25.8 (16.6) 28.3 (18.0) 31.1 (18.4) 23.5 (16.7) 25.1 (13.7) 21.1 (15.0) Percentage of female patients 49.7 (2.9) 49.8 (2.6) 49.6 (3.1) 49.4 (3.5) 50.0 (2.2) 50.3 (1.7) 50.0 (1.7) Percentage of patients aged 65 or over 15.4 (6.2) 15.0 (5.8) 15.4 (5.9) 14.3 (6.9) 16.8 (5.5) 19.6 (5.3) 16.3 (4.1) General Practitioner (GP) age 47.6 (7.5) 48.3 (7.9) 47.3 (7.8) 48.2 (8.5) 46.3 (6.4) 47.4 (6.2) 47.9 (7.3) Percentage of female GPs 43.5 (26.3) 39.5 (27.3) 38.6 (26.5) 38.9 (28.8) 45.9 (22.8) 37.3 (24.4) 42.1 (27.5) Percentage of UK qualified GPs 70.6 (35.3) 62.8 (37.2) 63.0 (37.7) 60.3 (39.4) 77.7 (29.9) 84.9 (27.4) 73.5 (34.3) Percentage of GP providers 74.0 (25.1) 73.4 (27.6) 73.0 (30.7) 72.2 (31.1) 72.0 (23.0) 78.1 (22.4) 79.1 (21.7) Percentage of Urban practices 82.90% 88.80% 83.90% 89.60% 85.30% 66.70% 84.80% Percentage of GMS practices 59.00% 59.00% 40.90% 54.00% 46.60% 65.40% 64.40% * Reporting means (sd) or percentages Kontopantelis Clinical Computing Systems and QOF 23. Background Methods Findings Summary Variation Model results Predictors of practice performance 2007-8 to 2010-11 Little change over time Most practice and patient characteristics had signicant but small effects for every additional 1000 patients on the practice list: RA decreased by 0.11%, PA by 0.14%, PQ increased by 0.08% practices in more deprived areas performed worse across all three outcomes: -0.01% for RA, -0.02% for PA and -0.006% for PQ, per 1 point increase on the IMD scale (equivalent to 0.8%, 1.6% and 0.48% in the most afuent compared with the most deprived areas) rural practices scored lower on RA and PQ but not on PA practices with a higher % of female patients performed better on all three outcomes Kontopantelis Clinical Computing Systems and QOF 24. Background Methods Findings Summary Variation Model results Predictors of practice performance 2007-8 to 2010-11 Overall performance differed signicantly by clinical system used, for all three outcomes Best: Vision 3 (RA), Synergy (PA & PQ) Worst: PCS, across all three measures Synergy practices on average gain 602 more than PCS practices each year Performance by type of activity varied signicantly across clinical systems, for all three outcomes Similar system rankings Biggest differences across outcome indicators Kontopantelis Clinical Computing Systems and QOF 25. Background Methods Findings Summary Variation Model results Predictions of practice performance, by system 2007-8 to 2010-11 Reported Achievement Population Achievement % of points scored Reported Achievement Population Achievement % of points scored Predictions (95% CI) rank Predictions (95% CI) rank Predictions (95% CI) rank Predictions (95% CI) rank Predictions (95% CI) rank Predictions (95% CI) rank Overall (aggregate over all 62 indicators), from regression model 1 Outcome (aggregate over 11 indicators), from regression model 2 Vision 3 90.1 (90.0,90.2) 1 85.4 (85.3,85.5) 2 97.8 (97.7,97.9) 3 81.0 (80.9,81.1) 2 75.3 (75.2,75.4) 2 98.3 (98.1,98.6) 1 Practice Manager 89.8 (89.5,90.1) 2 84.7 (84.4,85.0) 5 97.4 (97.0,97.8) 6 81.4 (81.0,81.7) 1 75.0 (74.6,75.4) 3 98.2 (97.8,98.7) 3 Synergy 89.8 (89.7,89.9) 3 85.6 (85.4,85.7) 1 98.1 (97.9,98.2) 1 80.4 (80.2,80.6) 4 75.4 (75.2,75.6) 1 98.0 (97.7,98.3) 4 Premiere 89.8 (89.5,90.0) 4 85.1 (84.8,85.3) 3 98.1 (97.8,98.4) 2 80.7 (80.4,81.1) 3 74.9 (74.6,75.3) 4 98.3 (97.9,98.7) 2 ProdSysOneX 89.6 (89.4,89.7) 5 84.5 (84.4,84.6) 6 97.6 (97.5,97.7) 5 80.3 (80.1,80.5) 5 74.1 (73.9,74.2) 6 97.8 (97.5,98.1) 5 LV 89.4 (89.3,89.5) 6 85.0 (85.0,85.1) 4 97.7 (97.6,97.8) 4 79.5 (79.5,79.6) 6 74.3 (74.2,74.4) 5 97.8 (97.5,90.0) 6 PCS 88.7 (88.6,88.8) 7 84.3 (84.2,84.4) 7 97.3 (97.2,97.4) 7 78.7 (78.5,78.8) 7 73.3 (73.2,73.5) 7 97.4 (97.1,97.7) 7 Kontopantelis Clinical Computing Systems and QOF 26. Background Methods Findings Summary Variation Model results Relative monetary gains, compared to PCS 2007-8 to 2010-11 602 563 379 308 219 79 -200 -100 0 100 200 300 400 500 600 700 Overall Measurement Treatment Outcome Kontopantelis Clinical Computing Systems and QOF 27. Background Methods Findings Summary Findings Performance levels differed across systems, even after controlling for practice and patient characteristics Differences were small in absolute terms, but: the association between system and performance was stronger than for any other patient or practice characteristic substantial at the population level e.g. 1% difference in achievement of BP control targets for HT patients 9 patients in average practice; over 71,000 patients nationally Overall, best average performance across all outcomes measures was by Vision 3, Synergy or Premiere systems Synergy & Premiere in a diminishing minority of practices and are now likely to be withdrawn from the market Remuneration differences were practically negligible Kontopantelis Clinical Computing Systems and QOF 28. Background Methods Findings Summary Conclusions QOF performance is partly dependent on the clinical computing system used by practices Raises the question of whether particular characteristics of computing systems facilitate higher quality of care, better data recording, or both Inconsistency across clinical computer systems which needs to be understood and addressed Researchers who use primary care databases, which collect data from a single clinical system, need to be cautious when generalising their ndings Messages relevant to international health care systems Kontopantelis Clinical Computing Systems and QOF 29. Appendix Thank you! Kontopantelis E, Buchan I, Reeves D, Checkland C and Doran T. The Relationship Between Quality of Care and Choice of Clinical Computing System: Retrospective Analysis of Family Practice Performance Under the UKs Quality and Outcomes Framework. BMJ Open. In print. This project is supported by the School for Primary Care Research which is funded by the National Institute for Health Research (NIHR). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. Comments, suggestions: e.kontopantelis@manchester.ac.uk Kontopantelis Clinical Computing Systems and QOF