For the Practice Change Fellows Program September 25, 2008 Washington, DC Dennis A. Ehrich, MD, FACC...
-
Upload
rodger-garrison -
Category
Documents
-
view
213 -
download
0
Transcript of For the Practice Change Fellows Program September 25, 2008 Washington, DC Dennis A. Ehrich, MD, FACC...
For the Practice Change Fellows ProgramSeptember 25, 2008
Washington, DC
Dennis A. Ehrich, MD, FACCVice President for Medical Affairs
St. Joseph’s Hospital Health CenterSyracuse, New York
The Importance of Measurement in Health Care
Agenda for the Afternoon
1-Why we measure in health care?2-The Model for Improvement3-Selecting one’s measures4-Time ordered statistics and understanding
variation 5-Displaying and tracking results6-Deciding whether To design a new process or
improve an existing process
Why We Measure in Health CareMeasuring for
ResearchMeasuring for
JudgmentMeasuring for Improvement
Purpose To discover new knowledge
To compare to others, to rank
To bring new knowledge into daily practice
Tests One large trial Public reporting quarterly or with 12
month running averages
Many small, sequential, observable tests
Bias Control for as many as possible
Severity or risk adjustment where
available
Stabilize the biases from test to test
Data Gather as much data as possible, just in
case
Measures structure, process or outcomes
Usually applied to process
Duration Can require large numbers of patients and long periods of
time to obtain results
Ongoing data collection and periodic
public reporting
Short iterative cycles in a limited number of subjects,
followed by spread
Set aims that are measurable, time-specific, and apply to a defined population
The Model for Improvement
Establish measures to determine if a specific change leads to improvement
Select changes most likely to result in improvement
Test the changes
T. Nolan et al. www.ihi.org
The Use of Iterative PDSA Cycles
Implementing the Changes
“Rapid-cycle CQI”
T. Nolan et al. www.ihi.org Multiple Simultaneous Tests of Change
Spreading the Change
1-Executive sponsorship2-Planning and set-up 3-Spread within the target population-social network theory 4-Continuous monitoring and feedback during the spread process5-Capturing and sharing organizational learning
T. Nolan et al. www.ihi.org
Donabedian’s Quality Triangle-It’s Relevance to Process Improvement
-Avedis Donabedian, MD, MPH (1919-2000)
Donabedian’s TriadStructure
OrganizationPeopleEquipment/Technology
ProcessThe steps taken in accomplishing the change and achieving the
outcomeResults must be client-focusedMust deliver results reliably
OutcomesClinical (mortality, complications)Client perception or satisfactionFinancial
The Three Domains of Measurement
• Structural Measures• Process measures• Outcomes Measures
– Balancing measures
Donabedian
The Three Domains of Measurement
• Structural Measures– Describe the environment. How many?– Square footage of a clinical unit– Number of staff– Staff qualifications and competencies– Presence or absence of technology and its
characteristics• Process Measures
• Process cycle time• The percentage of patients for whom the process achieves
its desired result
Donabedian
The Three Domains of Measurement
• Outcome Measures• The impact of the change initiative on mortality,
readmissions to the hospital, ED visits• The satisfaction scores of clients and staff • The cost per case, average LOS, revenue per case
• Balancing Measures – Unintended outcomes that are consequences of the
new program– Unanticipated mortality, morbidity or cost – Has the shifting of resources in an organization
compromised other client or patient populations?Donabedian
ACTION
Aim
Selecting A Measure
Operational Definitions
Data Collection Plan
Data Collection
Data Analysis
The Quality Measurement Roadmap
Modified from Lloyd, Robert: “Quality Health Care A Guide to Using Indicators”
Selecting a Measure:
-When selecting a measure, have clarity as to whether the measure is one of structure, process or
outcome
-And select a balanced panel of indicators that reflect the dimensions of performance being evaluated and the change concept(s) being
employed
What Dimension of Performance Do You Want to Measure?
• Appropriateness • Availability• Continuity• Effectiveness• Efficiency• Respect and caring• Financial/Viability• Safety• Time lines
Joint Commission (1996)
What Dimension of Performance do You Want to Measure?
• Safety• Effectiveness• Patient-centeredness• Timeliness• Efficiency• Equity
IOM: Crossing the Quality Chasm (2001)
What is the “Change Concept”?• Eliminate waste• Improve work flow• Shorten a waiting list• Change the work environment• Improve the Provider/Client interface• Manage time• Focus on variation• Error proofing a process• Focusing on product or service
The Improvement Guide by Langley, Nolan, Nolan, Norman and Provost. Jossey-Bass
Relating a Change Concept to a Specific Measure
Concept Potential Indicators for this processPatient scheduling •The average number of days between the call for an
appointment and the actual appointment date•The percentage of appointments made within 3 days of the call for an appointment•The number of appointments scheduled each day
Home care visits •The number of home care visits•The average time spent during a home care visit•The percentage of time spent traveling during each home care visit•The number of visits per home care nurse
CQI Training •The number of participants attending a class•The percentage of cancellations•The percentage of no-shows•The information recall scores at 30 and 60 days
Operational Definitions• Is clear and unambiguous• Specifies the measurement method, procedures and
equipment when appropriate– Clinical data (chart reviews) vs. administrative data– Client logs vs. a computer database
• Define specific criteria for the data to be collected– Define all inclusions and exclusions– For percentages or rates, or ratios, define the criteria
for inclusion in the numerator and denominator• Always ask “How might somebody be confused by this
definition?”
Lloyd, R. Quality Health Care (2004) Jones and Bartlett
Examples of Unclear Definitions
• Timely completion of the screening process• A complete medication list• The readmission rate• Medication error• Cost impact• From the acute care hospital
– A patient fall– Surgical start time
Lloyd, R. Quality Health Care (2004) Jones and Bartlett
Data Analysis
• How will the measurements be expressed?– Quantities, rates, ratios, proportions, percentages
• What type of statistics will be used?– Descriptive statistics
• Measures of central tendency– Mean, median, mode
• Measures of variation or spread– Minimum, maximum, range, standard deviation
– Inferential statistics• t-tests• ANOVA• Chi Square
Data Display
• Table• Bar chart• Histogram• Line chart • Pie chart• Pareto diagram• Time-ordered data
• Run chart• Control chart
Comparative Data
• Internal targets-trended data• External comparisons-benchmarking
– Best practices– National or regional population averages
External BenchmarkingJoint Commission
CMS
Calculation of the Confidence Interval
Estimates
± t * σ/ √n
Wheret= 3 (the sigma number for 99% confidence interval)
σ =The hospital’s standard error of the mean and
n = The number of patients in the hospital’s denominator
Data Reporting
• Data reporting plan– Who will receive the results?– How often will they receive the results?– How will it be formatted?
• Dashboard• Paper reports• Spider diagram
– How will the data be disseminated?• E mail• Internet• Intranet
Tools for Displaying Time-ordered Data
• Run charts– Plot of data over time with the median of the data
set plotted as a center line
• Control charts– Plot of data over time with the mean as the center
line and with upper and lower control limits
Run Charts
• Easily constructed by hand or in available spreadsheet programs
• Provides a good idea of improvement in a change initiative
• Less sensitive to significant changes (special cause variation) than the control chart
Control Charts More sensitive to special cause variation than a run
chart Requires specialized computer software to create There are 9 types of control charts used in health
care, depending upon whether the data collected is distributed normally, is continuous (numerical) or discreet (attributes) and whether the events measured are frequent or infrequent
Have their own set of rules to identify special cause variation
Understanding Variation• All data, collected over time, varies• Random variation (common cause)
– The changes occurring are intrinsic to the process being measured
• Non-random variation (special cause)– The changes are being imposed on the system by some external
factor– May be unintended and un anticipated or may be by design
• Before process improvement can be implemented, the process must be in control (free of special cause variation)
Special Cause Variation in a Control Chart
Upper Control Limit 205 mmHg
Lower Control Limit 142 mmHg
Mean 173 mmHg
Special Cause Variation 138 mm Hg
Daily record of Blood Pressure
Initial Considerations
• Is the process under consideration local?– Within a department– On a clinical unit
• Is the project organization wide?– A process change in a work system that impacts
the entire organization– Requires commitment of people, funds, or new
technologies
Organization-Wide Initiatives
• Must be consistent with the organization’s Mission, Vision, and Values
• Must be aligned with the organization’s strategic plan
Measurement and the Strategic Plan
Analyze the inputsObtain Inputs
Determine the organizational strategies for each strategic goal
Map the data sourceLocate or design the system Write the interfacesPopulate the dashboards
Determine the departmental tactics, measures, and targets
Determine HR Requirements Formulate the IT Capital Budget
Staffing requirementsGrow or PurchaseTraining requirements
Determine the organizational measures, performance Targets
and benchmarks