2C - Using Data to Improve - IHIapp.ihi.org/.../Using_Data_to_Improve.pdf · Lean Management System...
Transcript of 2C - Using Data to Improve - IHIapp.ihi.org/.../Using_Data_to_Improve.pdf · Lean Management System...
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
1
2C – Using Data to Improve
Advanced Measurement for Improvement Seminar
March 26-27, 2015
The Data Cycle
Measures identified
and defined
Data collection
process defined,
tested
A P
DS
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
2
Data Acquisition
Operational IT systems gather granular data on standard processes
� Clinical: Nursing, EHR, Labs, Pharmacy, etc.
� Administrative: Billing, scheduling, etc.
Supplemented by systems to gather clinical process data
� Institutional
� Ad-hoc
PDSA data is real-time, front-line, manual.
Interpretation and Application
Who needs to know what?
� What level of information
� How often? How soon?
Will the audience interpret the measures appropriately?
� How will you train them?
� How will you keep them consistent?
Will process owners know how to respond?
� How will you coach them?
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
3
Source: Virginia Mason Health System
Lean Management System
Ideal management system to support value-based production:
Leader standard work
Visual controls
Daily accountability and planning
Respect for people who do the work
Unity of purpose
Strategic intent, operational goals, and system views must be vertically aligned!
Mann, D. (2010). Creating a Lean Culture: Tools to sustain lean
conversions. Boca Raton, FL, CRC Press.
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
4
Systems Hierarchy
Macro-systems
e.g. trust, facility, region
Meso-systems
e.g. division, clinical dept, pathology, IT
Microsystems
e.g. unit, clinic, surgical team
Source: Virginia Mason Health System
“Catchball” process aligns levels
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
5
Reporting Improvement
Senior Leaders, Boards, Executive
Sponsors (Macro-system)
Percent of target sites engaged in key
improvement initiatives
Percent of target population exposed to interventions
Phase of intervention by site or project: Plan? Pilot?
Implementation? Spread?
Time-series family of key ‘current care’ and ‘population’
measures by site, with goals
Comparison to ‘best practice,’ national/regional datasets,
comparative benchmarks
Comparison to control sites
Source: Keith Mandel MD
Reporting Improvement
Improvement Initiative Leaders, Department
Heads, etc. (Meso-system)
Time-series dashboard of all
‘current care’
and ‘population’ measures by site, with goals.
Key current care measures segmented by unit, patient
sub-population, risk groups. Measures matched to
domain of improvement work.
Current QI capability of site leaders and teams, other
‘foundational’ requirements (e.g. registry, EMR)
Degree of involvement/effort of QI teams
Data quality
Source: Keith Mandel MD
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
6
Reporting Improvement
Front-Line (Micro-system) Teams
Time-series dashboard of all
‘current care’
and ‘population’ measures by site, with goals.
Key current care measures segmented by unit, patient
sub-population, risk groups. Measures matched to
domain of improvement work.
PDSA measures for current process change testing.
Data quality
Source: Keith Mandel MD
Exercise
For Your Own Project:
Identify the key data ‘customers’ and their relationship to (or role in) the project?
What is their degree of involvement in the project and familiarity with QI methods?
How can you leverage measurement to maximize their engagement in the work?
What information are they receiving now? Is it timely and accurate?
What are your ideas for improving data feedback?
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
7
Individuals or Groups Role in Project
Degree of
Involvement
(1=never – 5=daily)
Comprehension of
Methods and Goals Ideas for Engagement
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Individuals or Groups Role in Project
Degree of
Involvement
(1=never – 5=daily)
Comprehension of
Methods and Goals Ideas for Engagement
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Key Data
Customers
Currently Receiving
Information?
Time Lag,
Data Quality Ideas for Improvement
Percent of target sites engaged in key
improvement initiatives
Percent of target population exposed to
interventions
Phase of intervention by site or project:
Plan? Pilot? Implementation? Spread?
Time-series family of key ‘current care’ and
‘population’ measures by site, with goals
Comparison to ‘best practice,’
national/regional datasets, comparative
benchmarksComparison to control sites
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
Current QI capability of site leaders and
teams, other ‘foundational’ requirements
(e.g. registry, EMR)
Degree of involvement/effort of QI teams
Data quality
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
PDSA measures for current process change
testing.
Data quality
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Currently Receiving
Information?
Time Lag,
Data Quality Ideas for Improvement
Percent of target sites engaged in key
improvement initiatives
Percent of target population exposed to
interventions
Phase of intervention by site or project:
Plan? Pilot? Implementation? Spread?
Time-series family of key ‘current care’ and
‘population’ measures by site, with goals
Comparison to ‘best practice,’
national/regional datasets, comparative
benchmarksComparison to control sites
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
Current QI capability of site leaders and
teams, other ‘foundational’ requirements
(e.g. registry, EMR)
Degree of involvement/effort of QI teams
Data quality
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
PDSA measures for current process change
testing.
Data quality
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Are They Being
Served?
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
8
Monitoring the System of Care
• Alignment
• The measure ‘cascade’
• Strategic measure deployment
Dynamic & Static Views of a Process
0
10
20
30
40
50
60
70
80
90
100
3/1/2
008
3/8/2
008
3/15
/2008
3/22
/200
8
3/29
/200
8
4/5/
2008
4/12
/200
8
4/19/
2008
4/26/2
008
5/3/
2008
5/10/2
008
5/17
/200
8
5/24/2
008
5/31
/200
8
6/7/
2008
Control charts show
change over time
Histogram, radar charts,
etc. show cross-
sectional ‘snapshots’ at
a point in time 0
2
4
6
8
10
12
14
16
18
20
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
9
Caldwell, C. (1995). Mentoring Strategic Change in Health Care: An
Action Guide. Milwaukee, ASQC Quality Press.
Strategic Intent and Strategic Measures
Short Term – This year’s goals
� Cash flow & cost reduction
� Productivity, net revenue, receivable days
� Meet current clinical targets
� CHF readmits
Mid Term – Next year’s goals
� Increase market share
� Customer satisfaction, complaints
Longer Term – 3 year goals
� Increase organization agility
� # Improvement projects, improvement project cycle time
Caldwell, C. (1998). Results-driven management: Strategic quality deployment. The handbook for managing change in health care. C. Caldwell.
Milwaukee, ASQ Quality Press: 37-87.
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
10
Strategic Intent and Strategic Measures
Dimensions of system performance
Rate of innovation and improvement
Reduce non-value-added costs
Improve cash flow
Increase customer satisfaction
Progressively integrate the organization as a system (additional business units, standard practice, IT)
� Vertical
� Horizontal
Source: Caldwell, C. (1998)
West Paces Ferry Quality Dimensions c.1992
Productivity Sales Development Customer Loyalty
Source: Caldwell, C. (1998)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
11
West Paces Ferry Level 1 Measures c.1992
Productivity Sales Development Customer Loyalty
Cost per member per
month1
Target doctor recruits Net revenue from new
products
Days to resolve a
complaint
Cash flow percent
prior year (growth)
Corporate contracts QI projects completed Health status – quality
of life
Cost of poor quality Public awareness of
brand
Employee satisfaction
– open communication
Patient brag
Income percent prior
year
Market share QI project percent
complete
Operating expense
percent prior year
Readmit percent2
1WPF was an integrated delivery system 2Quality target for corporate strategy
Source: Caldwell, C. (1998)
Kano – Customer Judgment as a Basis for Performance Appraisal
Kano, N. (1984). "Attractive Quality and Must-Be Quality." Journal of the Japanese Society for Quality Control 14(2): 39-48.
I
II
III
III. Delightful. Unexpected and
exciting
II. Normal. A satisfactory
experience
I. Expected. Below this level repels
customers
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
12
Radar Chart: Quality Dimensions
I
II
III
Productivity
Productivity
Development
Sales
Patient BragCost of poor
quality
Source: Caldwell, C. (1998)
Suboptimized Systems
Source: Caldwell, C. (1998)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
13
Level 1 Radar Chart in Action
Source: Caldwell, C. (1998)
The Information Cascade
Macro-systems
e.g. system, trust, facility, region
Meso-systems
e.g. service line, division, clinical dept, pathology, IT
Microsystems
e.g. unit, clinic, surgical team
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
14
Levels of Measurement
1 - Strategic measures
• Derived from strategic dimensions
(e.g. Balanced Scorecard)
• Target current, mid, long term goals
• Align with strategic plan
2 - Division measures
• Structural units comprising key
organizational functions
• Most L3 are operational
‘management indicators’
3 - Business process indicators
• Measures of high-level process
effectiveness and efficiency
• Components may have different
owners
4 - Core mainstay and support process
indicators
• Single process owner
• This is where QI work is focused
(1)
(2)
(3)
(4)
(Levels)
Ma
na
ge
me
nt V
iew
Macro
Micro
Meso
Micro
Admin errors
per 100 scripts
Wrong patient
per 100 scripts
% errors
intercepted
Non-path orders
% cases
Allergy alerts
per 100 scripts
Medication
errors % dsch
Prescribing errors
per 100 scripts
Moving up:
• Cause-effect theory (e.g. driver diagram, clinical evidence)
• Observed correlation (e.g. regression models)
• Aggregation
Data flow to more macro levels
Management ‘line of sight’
‘Line of Sight’ Measures
Source: Caldwell, C. (1998)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
15
# Calls to rapid
response team
Environment
Hand hygiene
compliance
‘Line of Sight’ Measures
Percent
inpatient
mortality
Compliance with
“bundles”
% Surgical
bundle
% Pressure
ulcer bundle
% CL bundle
% VAP bundle
Hospital
Acquired
Infection
rates
% Sepsis
bundle
L1 L2 L3 L4
AggregationDriver Model
Observed
correlation,
clinical
evidence
Aggregation Methods
• Individual Patient Data to Population
� Average, median, distribution of patients: Cost, Time, Scores,
etc.
� Percent conforming: Protocol-driven care
� Count of events: Falls, Mortality, ADEs, etc.
• Micro to Meso to Macro
� Numerators and denominators summed across units
� Overall averages, medians
� Average unit performance
• Aggregating Across Different Measures
� Staging systems
� Build composite measures or indices
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
16
Staging System
Griffin, F. A. and D. C. Classen (2008). "Detection of adverse events in surgical patients using the Trigger Tool
approach" Qual Saf Health Care 17(4): 253-258.
Discussion
Consider how the aim of your project fits into your organization’s strategic goals:
Do the key measures that track the success of your project fit into a measure cascade within the organization? What would that look like?
Do you have recommendations for your client regarding a strategy for operational measurement?
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
17
Dashboards
Examples
Why not ‘Red-Yellow-
Green’?
An ideal alternative
© 2009 Institute for Healthcare Improvement, R. Lloyd
Tables require perusal
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
18
“Dashboard” = Summary of Performance Measures
“Radar” chart provides a snapshot view of multiple key quality indicators for a single unit:“Where are we right now?”
Source: Caldwell, C. (1998)
© by R. Scoville
“Small multiples” view compares many units on a single dimension (& over time):“What changes do we see, and where?”
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
19
Bed Occupancy Dashboard
Picture of ER display system to control be utilization
Source: Rostow (2002)
Source: Provost, Murray & Britto (IHI Forum 2010)
‘Traffic light’ emphasizes goals…But more about this later….
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
20
Who Uses Hospital Dashboards?
Survey: “Who is given the scorecard you supplied to us, and how frequently?” N=139
Kroch et al. (2006)
Who Uses Hospital Dashboards?
“Shorter, more focused dashboards that are reviewed on a
frequent basis are associated with higher performance.
According to the results of this dashboard analysis,
hospitals that use dashboards with fewer measures are
more likely to be in the high-performance group, suggesting
that higher-performing hospitals have developed
dashboards that focus on areas they see as critical for
quality. Furthermore, performance data are more
actionable when such data are consistently reviewed by the
board on a relatively frequent basis.”
Kroch et al. (2006)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
21
A Common Type of Dashboard
Source: Provost, Murray & Britto (IHI Forum 2010)
This ‘specifications’ view does not provide a predictive view of system dynamics
How Is Error Rate Doing?
Source: Provost, Murray & Britto (IHI Forum 2010)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
22
How is Perfect Care Doing?
Source: Provost, Murray & Britto (IHI Forum 2010)
Alternative
A view where
Each measure is displayed on an appropriate control chart
All control charts are on same page to see the whole system
Advantages
More accurately assess meaning of system changes
Become aware of system interrelationships
Appreciate dynamic complexity
Base decisions for action on improvement signals
HOWEVER…
Requires the viewer to understand variation!
Source: Provost, Murray & Britto (IHI Forum 2010)
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
23
Control Chart Dashboard
Source: Provost, Murray & Britto (IHI Forum 2010)
SPN Dashboard Report Fall 2010
Advanced Measurement for Improvement
Cambridge, MA • March 26-27, 2015
24
Small Multiples: One site, all measures
Source: Dentaquest Institute
Small Multiples
One measure, all
sites
Source: Dentaquest Institute