Post on 25-Dec-2015
1 04/19/23 07:38 LifePaths
On Measuring Outcomes and Productivity in Canada’s
Health Care Sector first principles and basic definitions the UK Atkinson Report, the System of
National Accounts (SNA), and the “standard” approach to measuring (health sector) productivity
empirical nuggets and “awkward facts” alternative and better approaches
Michael Wolfson
Statistics Canada
NB – More detail can be seen in the “notes view”; still draft, please do not circulate without permission
2 04/19/23 07:38 LifePaths
First Principles - I
Population health is the fundamental objective Health outcomes relate to / depend on health
interventions• i.e. social activities, whether deliberate or
inadvertent Health care is one kind of intervention which often contributes to health• but not always, and certainly not solely
3 04/19/23 07:38 LifePaths
First Principles - II
Doing more with less is a good thing• i.e. being more efficient or more
productive is beneficial People and care providers are
heterogeneous• so summing or averaging to produce
overall indices can produce misleading results
4 04/19/23 07:38 LifePaths
Definition (by Construction) - Population Health I
ask everyone (or a sample thereof) a structured set of questions (or do an exam) to assess each person’s health status• i.e. a profile for each person (n.b. gives micro
detail) construct an index for each person, based on
their health profile• e.g. McMaster Health Utility Index, or QALY
average over people (perhaps age-standardized)
5 04/19/23 07:38 LifePaths
Definition - Population Health II
or combine individual-level summary health indices with life table (mortality rates) to measure Health-Adjusted Life Expectancy (HALE)
6 04/19/23 07:38 LifePaths
“Cause – Deleted” Changes in Life Expectancy (LE) and Health-Adjusted Life
Expectancy (HALE)
2.4
1
0
0.5
0.4
0.4
0.3
0.1
0
0.7
1.8
0.7
0.6
0.8
0.3
0.4
0.3
0
0
0.4
00.511.522.5
IHD
Lung cancer
Breast cancer
Stroke
COPD
Colorectal cancer
Diabetes
Melanoma
Osteoarthritis
Mental disorders
Men
Women
Source: Manuel et al, ICES and Health Canada, NPHS
HALELE
2.2
0.9
0.7
0.5
0.4
0.5
0.2
1
0.9
1.5
0.6
0.5
0.7
0.5
0.3
0.4
0.1
2.4
1.1
0 0.5 1 1.5 2 2.5
Men
Women
7 04/19/23 07:38 LifePaths
Definition - Health Outcome
health status “before”
health status “after”
health intervention
other factors
health outcome change in health status attributable to a health intervention
(for an individual)
8 04/19/23 07:38 LifePaths
Charles Wright on Vancouver Cataracts pre- and post-surgery patient self-completed questionnaires “31% of patients booked for cataract surgery report a visual
function score of 91 points or more on a scale of 100. … “These data tend to confirm the observation that cataract
surgery is now occurring in many patients with minor degrees of self-reported visual disability. …
“The overall results are positive, but 27% of patients show either no change or deterioration of VFA (Visual Function Assessment) score after the operation.”
9 04/19/23 07:38 LifePaths
Relative Risks of “Preventive” Tamoxifen0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
(9.27)
(Fisher et. al., J National Cancer Institute, 1999)
Breast Ca
Fractures
CHD
Cataracts
Stroke
Deep Vein Thromb
Endometrial Ca
Pulmonary Emb
10 04/19/23 07:38 LifePaths
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0.12
1.66 2.08 2.49 2.91 3.32 3.74 4.15
5-yr Predicted Risk / Proportion of Women Affected
Ch
an
ge
in
Lif
e E
xp
ec
tan
cy
Simulated Change in Life Expectancy for Canadian Women for Alternative Scenarios of Preventive Tamoxifen
(95% CIs)
42.3% 24.6% 16.2% 9.0% 4.0% 2.2% 1.7%
(Will et. al., British J Cancer, 2001)
11 04/19/23 07:38 LifePaths
Definition – Health Care, per the Evans and Stoddart “Plumbing Diagram”
“bottom line”
“thermostat”
12 04/19/23 07:38 LifePaths
(Tu et al on Coronary Surgery)
n.b. virtually no differences in one year survival; but no data on differences in health-related QoL
e.g. almost 17x, with no
benefits?
13 04/19/23 07:38 LifePaths
0
2
4
6
8
10
12
14
16
18
0 20 40 60 80 100 120
Mortality Rate(%, age 65-70)
Career Earnings and Deathfor 500,000 Canadian Men
top quintile
(Career Earnings and Death)
Average Earnings (age 45-64, 1988 $000s)
Source: Wolfson et al., Gerontology, 1993
14 04/19/23 07:38 LifePaths
Definition – Productivity (General)
productivity level “output” / “input” productivity growth growth in outputs - growth in
inputs i.e. getting more output for given inputs, or getting the
same output from fewer inputs n.b. in common parlance
• no presumption that everything has to be measured in $$$• indeed, usual thoughts are in physical units (e.g. patients seen
or cataracts done per day)
15 04/19/23 07:38 LifePaths
Definition – Productivity (business school)
let me compare myself to another firm, typically a competitor
is she producing her widgets at lower unit costs than me?
i.e. benchmarking for individual product lines
16 04/19/23 07:38 LifePaths
Definition – Productivity (“standard” economics and SNA)
the economy has myriad productive agents (firms)• each of whom uses inputs = total capital services + total labour
services (factors of production)• to produce outputs (goods and services) summing to GDP
everything is measured in $ -- with the total being (conceptually) the sum of unit prices x quantities• but over time, prices (p’s) change, and this is not “real” • and quantities (q’s) change e.g. in terms of “quality”
to measure productivity, time series of outputs and inputs are constructed• taking out “pure” price changes, and• adjusting for improvements in quality• so that productivity = output – sum { inputs }
17 04/19/23 07:38 LifePaths
Definition – Health Sector Productivity (“standard” economics and SNA)
“standard” economists and National Accountants want to treat “health care” as an industry, with “health care services” its outputs, analogous to private sector industries
“firms” in the health sector are divided (mainly) into hospitals, nursing homes, and providers of ambulatory care (OECD SHA)• n.b. no concept of “regional health authority” or “local health
integration network”
ideal concept for “outputs” is care for “episodes of illness”, though DRGs in practice (OECD SHA)• n.b. no concept of “continuum of care”, nor chronic illness, nor
recognition of co-morbidities
18 04/19/23 07:38 LifePaths
Public Sector Challenge to “Economists’ Productivity”
???“Profits”
Inputs
Commercial Sector
Public Sector
Outputs
19 04/19/23 07:38 LifePaths
UK Office of National Statistics (ONS) and their “Productivity Paradox”
UK Labour Government massively increased spending in health care (and education) starting in the late 1990s
the ONS had been dutifully measuring public sector productivity, using SNA / economists’ concepts
productivity declined (unfortunately) ONS (2004) asked Sir Tony Atkinson what to do (and many jurisdictions are considering the Atkinson
report recommendations, as well as gradually adopting the OECD’s System of Health Accounts)
20 04/19/23 07:38 LifePaths
ONS, Atkinson, and Productivity mandate from ONS National Statistician:
• “To advance methodologies for the measurement of government output, productivity and associated price indices” (OK)
• “in the context of the National Accounts” (Oh oh!) question: why not first pose issue in general and then only secondarily ask whether SNA is an
appropriate framework, and if not what would be?
21 04/19/23 07:38 LifePaths
Atkinson Report – Analysis I SNA data are the essential foundation
• for macroeconomic management• and as an indicator of social welfare
UK context per Bank of England (May 04)• CPI up 10% from 1997Q1 to 2003Q4• nominal government spending up 62%• ONS measure of real public sector output up 14% (Huh?)
GDP as welfare measure• more $ on (e.g.) health care treatments increases welfare –
certainly if appropriate and effective, but “asymmetric information”
22 04/19/23 07:38 LifePaths
Atkinson Report – Analysis II “National Accounts are not a substitute for
performance indicators” (para 1.27) “It is not necessarily the case that even a crude
measure of government output is preferable to an index based on total cost.” (para 2.25) – i.e. the conventional way of doing the SNA
measurement of quality change (e.g. improvements in methods and technology) is a major challenge
the UK, as part of the EU, is bound to measure SNA according to international standards (but the US and Canada, so far, have ignored these)
23 04/19/23 07:38 LifePaths
Atkinson Report – Selected Recommendations
the SNA should measure government non-market output (e.g. health care services) using a procedure parallel to that of the market sector (para 4.7)
use the “treatment” or GP visit as the canonical “output” of the health care sector
weight different kinds of treatment by their costs try to adjust for quality change – ideally by moving from
treatments to “care pathways”, and connecting care to health outcomes• n.b. sounds good, but feasibility?
also treat shorter waits as improved quality• n.b. nothing on appropriateness, or “watchful waiting”
24 04/19/23 07:38 LifePaths
“Technology Adoption from Hybrid Corn to
Beta Blockers”, Skinner and Staiger,
NBER, 2005
corn
corn corn
tractors
computers beta blockers
(Skinner I)
25 04/19/23 07:38 LifePaths
(education and) social capital
beta blockers
beta blockers
adjusted 1 yr mortality rate
$$$(Skinner II)
adjusted 1 yr mortality rate
(education and) social capital
adjusted 1 yr mortality rate
26 04/19/23 07:38 LifePaths
(Skinner III)
(Source: Skinner, Staiger, Fisher; Medical Technology, 2006)
27 04/19/23 07:38 LifePaths
“Wall of Ignorance”
28 04/19/23 07:38 LifePaths
Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions
0
5
10
15
20
25
0 10 20 30 40 50 60
percent revascularized within 30 days
percent dead
within one year Alberta
Quebec
British ColumbiaOntario
29 04/19/23 07:38 LifePaths
E. A. Codman and W.E. Deming Codman: early 1900s Boston surgeon famous for “End Results Cards” – to keep
track of surgical patients and follow them up one year later to • observe outcomes
• systematically learn from experience
100 years later: not yet implemented in health care
Deming: post WW II concern with product quality in manufacturing
father of field of statistical process quality control
50 years later: not yet implemented in health care
30 04/19/23 07:38 LifePaths
Underlying Person-Oriented Information (POI) for Heart Attack / Revascularization Analysis
Heart Attack (AMI)Treatment (revascularization = bypass or angioplasty)Death
one year observation window
one year follow-up window(excluded)
31 04/19/23 07:38 LifePaths
Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions
0
5
10
15
20
25
0 10 20 30 40 50 60
percent revascularized within 30 days
percent dead
within one year Alberta
Quebec
British ColumbiaOntario
32 04/19/23 07:38 LifePaths
Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions
0
5
10
15
20
25
0 10 20 30 40 50 60
percent revascularized within 30 days
percent dead
within one year
better (less intervention, & better survival)
Alberta
Quebec
British ColumbiaOntario
(more output ???)
33 04/19/23 07:38 LifePaths
Hospital 65+ Patient Co-morbidity
CHFHigh BP CPD Diab's Ca
RA etc. Psych Deprn
number (000's) 111 237 128 125 101 16 20 30
pct of all 16.4 35.0 18.9 18.5 14.9 2.3 3.0 4.5
cond'n only (%) 23.7 37.7 28.0 22.8 47.8 27.7 26.0 24.0
cond'n +1 37.1 37.6 38.0 41.9 31.0 36.3 35.1 35.0
cond'n +2 27.5 18.4 23.7 25.2 15.2 23.5 24.6 25.6
cond'n +3 9.9 5.4 8.7 8.5 4.8 9.6 10.6 11.6
based on 676,508 hospital inpatient discharges across 10 provinces in 2001/2
34 04/19/23 07:38 LifePaths
Health Care Outputs or Health Outcomes?
SNA approach: health care inputs health care outputs (i.e. “treatments”)• leave for others to figure out connections from health care outputs
health outcomes (para 7.27, OECD SHA)
public policy priority: what (broad) allocation of resources produces the most “health gain” (i.e. increase in population health) – inputs outcomes• SNA approach is helpful on inputs and costs• though focus on aggregation distracts from “benchmarking”, i.e. “firm”
level analyses• and SNA compulsion to create an artificial concept of “output” is useless
for this purpose
35 04/19/23 07:38 LifePaths
Platitudes?
You can’t manage what you can’t measure
You get what you measure
“Don’t ask how many (health care) events per pound; ask how much health per pound.” D. Berwick, BMJ 2005
36 04/19/23 07:38 LifePaths
(Ontario Framework)
Ontario Health Scorecard
production and use of evidence
sensible allocation
access / waits
continuity of care
health status
clinical results
healthy living
spending / resources
sustainability / equity
37 04/19/23 07:38 LifePaths
Vision – Coherent, Integrated Statistical System
Broad Summary Indicators
Regional Indicators / Planning Info
Facility Management Information / Unit Costs
Basic Encounter Data / Health Surveys
Health Accounts / Simulation Models