Presenting Data & Standardized Infection Ratios
-
Upload
sylvester-haynes -
Category
Documents
-
view
38 -
download
2
description
Transcript of Presenting Data & Standardized Infection Ratios
Presenting Data&
Standardized Infection Ratios
Tim Wiemken PhD MPH CICAssistant Professor of MedicineAssistant Director of Epidemiology and BiostatisticsUniversity of LouisvilleSchool of Medicine, Division of Infectious DiseasesClinical and Translational Research Support [email protected]: 502.852.4627www.ctrsc.net
Pre-Presentation Comment
• Do not collect any data you do not plan to use (unless your boss tells you to).
Pre-Presentation Comment
• What constitutes ‘use’?1. Displaying somewhere other than just a
meeting
2. Preliminary to a follow-up to collect data that will be used
3. Education
4. Targeted prevention
5. Surveillance (only if you educate with it!)
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
What Makes a Good Report?
The catch all answer – it depends!
There are some key ideas you should think about when preparing any report
What Makes a Good Report?
Create reports that can be USED, not just looked at.
…more later
What Makes a Good Report?
1. How much time do you have?
What Makes a Good Report?
2. Who is your audience?• How sophisticated are they?• What do they already know?• Remember that people think and speak differently
What Makes a Good Report?
3. Why are you giving the presentation?• To show data? (I hope not!)• To provide the status of something?• To prove your worth?• To affect change?• To get someone fired?
What Makes a Good Report?
Each of these helps you decide one key piece of information:
What do you NEED to present?
What Makes a Good Report?
A few rules:
1. Never present more than you need to present.
What Makes a Good Report?
A few rules:
1. Never present more than you need to present.
2. Never present less than you need to present.
What Makes a Good Report?
A few rules:
1. Never present more than you need to present.
2. Never present less than you need to present.
3. Keep it as short as possible.
What Makes a Good Report?
Therefore, focus on what is necessary to meet your goals.
What Makes a Good Report?
Yeah, so smarty pants, what is necessary?
What Makes a Good Report?
IT DEPENDS!
What Makes a Good Report?
Although what is needed depends on how much time you have, your audience, and your goal:
1.It is rarely OK to present raw data by itself:
What Makes a Good Report?
Infection Prevention and Control Report, May 2012:
CLABSI: 5
VAP: 4
CAUTI: 7
MRSA: 10
What Makes a Good Report?
Infection Prevention and Control Report, May 2012:
CLABSI: 5
VAP: 4
CAUTI: 7
MRSA: 10
What Makes a Good Report?
Infection Prevention and Control Report, May 2012:
This is only slightly better.This is only slightly better.
What Makes a Good Report?
Although what is needed depends on how much time you have, your audience, and your goal:
2.It is rarely OK to present significant amounts of text
– E.g. policies, procedures, etc.
What Makes a Good Report?
2. It is rarely OK to present significant amounts of text
– No one wants a six page policy handed to them in a meeting to review.
– It will not get reviewed (at least not well).– Send these out early… and many times.
What Makes a Good Report?
The Holy Order of Data Quality (worst to best).
1.Text2.Raw numbers3.Raw rates4.Tables of chronological numbers or rates5.Charts (hierarchy of charts later…)6.Charts with #47.Charts with #1, #3, and more fun statistics
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
Choosing a Chart
• Charts– Numbers alone are bad
Choosing a Chart
• Charts– Visual depiction is easier to quickly
comprehend
Choosing a Chart
• Charts– Visual depiction is easier to quickly
comprehend– Choosing the correct chart is extremely
important
Choosing a Chart
• Charts– Visual depiction is easier to quickly
comprehend– Choosing the correct chart is extremely
important– Some charts show trends or comparisons in
data
Choosing a Chart
• Charts– Visual depiction is easier to quickly
comprehend– Choosing the correct chart is extremely
important– Some charts show trends or comparisons in
data– The best chart does not have to be explained
Choosing a Chart
Choosing a Chart
Choosing Charts
• Pie– Less appropriate than one would think– Categorical data that comprise of portions that
add to 100% – Never use 3D pie charts – Do not use a pie chart if you have a lot of
categories
Choosing Charts
• Pie– Do not use a pie chart if you have a lot of
categories
Choosing Charts
• Pie
“Use a pie chart when you don’t have anything to say”
- Dr. Julio Ramirez
“Use a pie chart when you don’t have anything to say”
- Dr. Julio Ramirez
Pie Charts
No!
Choosing a Chart
Choosing Charts
• Bar– Snapshots of data [rates in one month]– Comparing data across different categories– Never use 3D charts unless you have a 3rd
dimension!
Bar Charts
No!
Bar Charts
• Is it ever appropriate to use 3D charts?
Yes! (Really?)
Choosing a Chart
Choosing a Chart
• Line – Longitudinal data (data over time)– Contiguous points
Choosing a Chart
• Line – Longitudinal data (data over time)– Contiguous points– Rates, Counts, etc. by Month/Quarter/Year
• Run charts• Statistical Process Control charts
Choosing a Chart
• Run charts– Use when you have few time periods (e.g.
<25 months).
Choosing a Chart
• Run charts– Anatomy
• Center Line / Median –represents the median of all of the data points.
• X-axis –represents the time period of interest (days, weeks, months, quarters, years).
• Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection).
• Data points – the actual data values.
Choosing a Chart
• Run chartsY-axis (Rate)
X-axis (Month)
Center Line (Median)
Data Points (<25)
Choosing a Chart
• Run charts– Use
• Use to identify when the data are different than you expect (for better or worse) through detecting abnormal variation
Choosing a Chart
• Run charts– Rules for abnormal variation
1.Seven or more consecutive points on either side of the Center Line (median).
2.Five or more consecutive points increasing or decreasing.
3.Fourteen or more consecutive points alternating up and down.
Choosing a Chart
• Run charts
Rule 17 points below median
Rule 25 consecutive points increasing
Choosing a Chart
• Statistical Process Control (SPC) charts– Use when you have many time periods (e.g.
≥25 months).– These are much better than run charts.
Choosing a Chart
• SPCcharts– Anatomy
• Center Line / Mean–represents the average of all of the data points.
• X-axis –represents the time period of interest (days, weeks, months, quarters, years).
• Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection).
• Data points – the actual data values.• Standard deviation lines (control limits) – represent 1,
2 or 3 standard deviations on each side of the center line
SPC Chart Components
MeanMeanMeanMean
SPC Chart Components
+1 SD +1 SD +1 SD +1 SD
-1 SD-1 SD-1 SD-1 SD
SPC Chart Components
+2 SD +2 SD +2 SD +2 SD
-2 SD-2 SD-2 SD-2 SD
SPC Chart Components
+3 SD +3 SD +3 SD +3 SD
-3 SD-3 SD-3 SD-3 SD
Upper Control Limit (UCL)
Lower Control Limit (LCL)
Types of SPC Charts
Choosing the correct chart can be difficult - it is also very important.Control limits are calculated differently for each chart....although they are marked in standard deviations, the formulas vary...
Types of SPC Charts
Choosing a Chart
• SPC charts– Use
• Same as Run Charts, except• Identify different types of variation
Variation
Common Cause - ‘ In Control’Common Cause - ‘ In Control’
Special Cause - ‘Out of
Control’
Special Cause - ‘Out of
Control’
Rules to Detect Special-Cause Variation
• one point above or below 3SD
• two of three points above/below 2SD
• four of five points above/below 1SD
• eight points in a row on either side of the mean
• trends of 6 points in a row increasing or decreasing
• fifteen points in a row within 1SD
• fourteen points in a row alternating up and down
• eight points in a row outside of 1SD
Variation
Common Cause - ‘ In Control’Common Cause - ‘ In Control’
Special Cause - ‘Out of
Control’
Special Cause - ‘Out of
Control’
A Special SPC Chart
• Often it is useful to provide reports that identify the ‘time since the last event’.
• This motivates and provides some competition
A Special SPC Chart
• Rare events make terrible charts.
• Rates for rare events do not show you much.
A Special SPC Chart
• You can combine the benefits of the ‘time between events’ and the issues with rare events into an SPC g Chart.
C chart (number of events)
g chart (time between events)
Choosing a Chart
Choosing Charts
• Radar– Rarely appropriate – confusing– May work for showing multiple percentages
in one small chart – Bar chart may still be better.
Radar Charts
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
Components of Good Charts
• Good charts tell the complete picture of what they contain
Components of Good Charts
• Good charts tell the complete picture of what they contain
• Ideally, someone who is not familiar with the data being presented should be able to easily and quickly understand what the chart is representing
Components of Good Charts
• Never use 3D charts unless you have a 3rd dimension
Components of Good Charts
• The scale used for charts (that have a scale) should begin at zero and end near the highest value– Changing the axis scale can tell a drastically
different story to those that are just glancing
Same Data, Different Scale
What would you say about these two services?
MRSA colonization rate by service
Same Data, Different ScaleMRSA colonization rate by service
MRSA colonization rate by service
How about now?
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
Tips for Reports
• Title must be very specific– Include person (organism/disease
state/compliance measure, etc.), place and time
• Example:– Hospital-associated Methicillin-resistant
Staphylococcus aureus Isolates: Hospital X, Acute Care ICU, January 2007-January 2010
Tips for Reports
• Data definitions should be included as a footnote or on an introductory page.
Tips for Reports
• Data definitions should be included as a footnote or on an introductory page.
• Example: Clostridium difficile– Definition for numerator
• Are you measuring toxin only? Antigen? By PCR? On liquid stool only?
– Definition for denominator• Patient-days? Admissions?
Example introductory slide for MRSA rates
Hospital-associated Methicillin-resistant Staphylococcus aureus
(MRSA)
Case of MRSA (Numerator): A case of MRSA was defined as a new and unique, hospital-associated (isolated >48 hours after admission), microbiological isolate from a patient admitted to hospital x during the month of interest without a prior history of MRSA.
Patient days (Denominator): The denominator for the calculation of the rate of MRSA was defined as the number of patient-days for hospital x during the month of interest, regardless of risk status.
Rate: (Numerator / Denominator) * 1,000
Tips for Reports
• For charts, you should also include a formula for how the rate was calculated.
Tips for Reports
• For charts, you should also include a formula for how the rate was calculated.– # Unique, hospital-associated MRSA
isolates divided by # Patient-days of care X 1,000
Tips for Reports
• For charts, you should also include a formula for how the rate was calculated.– # Unique, hospital-associated MRSA
isolates divided by # Patient-days of care X 1,000
– Be specific. Isolates, Infections, Colonizations, etc.
Tips For Reports
• Include the appropriate chart!
UCL
0.004
CL 0.001
0.0000
0.0005
0.0010
0.0015
0.0020
0.0025
0.0030
0.0035
0.0040
0.0045
Ra
te P
er
Be
d-d
ay
of
Ca
re
Date
Tips for Reports
• Raw data is important to be able to understand what is in the chart and to verify the data.
Tips for Reports
• Raw data is important to be able to understand what is in the chart and to verify the data.
• Some people just like the numbers.
Tips for Reports
• Raw data is important to be able to understand what is in the chart and to verify the data.
• Some people just like the numbers.
• Raw data should include numerator, denominator and standardized rate.
Tips for Reports
• Raw data is important to be able to understand what is in the chart and to verify the data.
• Some people just like the numbers.
• Raw data should include numerator, denominator and standardized rate.
Tips for Reports
• Including aggregated data for quick comparisons is also a good idea.
• Yearly average rates may work.
• When possible, include P-values to detect statistically significant differences (rate comparison example to come..)
Average Rate per 1000 Bed-days of Care 2008
Average Rate per 1000 Bed-days of Care 2009
Average Rate per 1000 Bed-days of Care 2010 YTD
1.3 1.7 0.7
Tips for Reports
• Always include a description of the assessment of the chart and your plans.
Tips for Reports
• Always include bubbles or call-out boxes that adequately describe dates and components of any interventions.
Tips for Reports
• You may use a bubble to describe if your surveillance definition changes.
Tips for Reports
• You may use a bubble to describe if your surveillance definition changes.
• A better idea is to just start over.
UCL 0.003
CL 0.001
0.0000
0.0005
0.0010
0.0015
0.0020
0.0025
0.0030
0.0035
0.0040
0.0045
0.0050
Ra
te P
er
Be
d-d
ay
of
Ca
re
Date
June 2006:Rate peaked to 4.30. Suspect increase was due to a change in MRSA precautions implemented affected staff compliance with when to wear PPE.
Hospital-Associated Methicillin-resistant Staphylococcus aureus Isolates: Hospital X, MICU, January 2006 – January 2010
# HA MRSA Isolates divided by # Bed-days of Care
Date
Number of Isolates
Number of Bed-Days of Care
Rate Per 1000 Bed-days of Care
Jan 10
Feb 10
Mar 10
Apr 10
May 10
Jun 10
Jul 10Aug 10
Sep 10
Oct 10
Nov 10
Dec 10
0 2
1319 1117
0.0 1.8
Average Rate per 1000 Bed-days of Care 2008
Average Rate per 1000 Bed-days of Care 2009
Average Rate per 1000 Bed-days of Care 2010 YTD
0.4 0.6 0.9
Assessment: Process is in statistical control.
Plan: Continue surveillance activities.
Tips for Reports
• Some reports must be text, although every attempt to transform into a graphic should be made.
Tips for Reports
• Some reports must be text, although every attempt to transform into a graphic should be made.
• Keep text to a minimum with summary information presented.
Tips for Reports
• Some reports must be text, although every attempt to transform into a graphic should be made.
• Keep text to a minimum with summary information presented.
• Email out early and often.
Tips for Reports
• Another option is a dashboard.
• This may be particularly useful for providing back to staff to post in their work areas.
• After all – a report that isn’t used for something other than looking pretty is not particularly worth your time!
Tips for Reports
Tips for Reports
• KHA has a wonderful template for you!
Overview
• What makes a good report?
• How to choose a chart
• Components of good charts
• Tips for reports
• Standardized Infection Ratios
Standardized Infection Ratio
• SIR
• A way to standardize/risk adjust rates for better comparability across institutions
Standardized Infection Ratio
• SIR
• Why are rates not comparable?– Different types of institutions have different
patient populations– These patients have different risks. – Therefore, comparing a University Hospital
MICU to a 4 bed Cardiac Unit in Wyoming is not useful.
Standardized Infection Ratio
• SIR
• SIR takes into account the types of patients seen in various different types of units or wards in a healthcare facility.
Standardized Infection Ratio
• SIR
• Compares the number of infections you have to the number you expect to see, based on some benchmark rate.
Standardized Infection Ratio
• SIR– SIR>1 : Worse than expected numbers– SIR <1 : Better than expected numbers– SIR = 1 : As expected
Standardized Infection Ratio• Advantages
– Adjusts based on benchmarked rates
– Adjusts for various locations [ICU vs Ward] or types of surgeries, etc.
– Excellent for comparing low numbers [removes high variation]
– Greater precision for adjustment
• Disadvantages
– Statistical bias = may not make appropriate comparisons
– Utility beyond SSI is relatively unstudied
– Wide understanding is just not there.
ICHE April 2006, 27(4): 427-429. ICHE January 2005, 26(1):8-9.
Standardized Infection Ratio
SIR will NEVER replace rates, particularly for interhospital comparisons (e.g. trends in your facility).
Their main use is for interstate comparisons and comparisons between facilities in a healthcare network.
SIR for CLABSI
1. Get observed numerator as usual
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days, vent-days, etc) as usual
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days, vent-days, etc) as usual
3. Get NHSN benchmark rate from the relevant publication
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days, vent-days, etc) as usual
3. Get NHSN benchmark rate from the relevant publication
4. Multiply benchmark rate by the number of patient days (line days, ventilator-days, etc)= expected number of infections
SIR for CLABSI
Divide your observed number by the expected number = SIR!
SIR for CLABSI
Example1.Get observed number of CLABSIs as usual: 5
SIR for CLABSI
Example1.Get observed number of CLABSIs as usual: 5
2.Get number of line-days as usual: 1,234
SIR for CLABSI
Example3.Get NHSN benchmark rate from the relevant publication
Can compare to different percentile
rates if that is your goal!
SIR for CLABSI
Example1.Get observed number of CLABSIs as usual: 52.Get number of line-days as usual: 1,2343.Get NHSN benchmark rate from the relevant publication: 1.9 4.Multiply benchmark rate by the number of line-days and divide by 1,000 = expected number of infections:
(1.9 *1,234) / 1,000 = 2.35
SIR for CLABSI
ExampleDivide your observed number by the
expected number = SIR!:
Observed (5)/ Expected (2.35) = 2.13
SIR for CLABSI
Example
SIR = 2.13
You are doing worse than expected given the benchmark.
SIR for CLABSI
• This method calculated an SIR for one particular unit.
• You can calculate an overall SIR that adjusts for multiple types of units. This makes it more comparable to other institutions.
Lets see how this works…
SIR for CLABSI
Example Using Free Software
http://innovationsforglobalhealth.com/ipstat/
Standardized Infection Ratio and SPC
• SIRs can be plotted on an XmR chart and may be more sensitive to detect special causes (very much controversy here!)
Final Thoughts
• Charts are good – use them whenever possible.
Final Thoughts
• Charts are good – use them whenever possible.
• Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in.
Final Thoughts
• Charts are good – use them whenever possible
• Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in.
• SIRs may be useful in some situations
Final Thoughts
• Make sure everyone knows how to read your charts
POST YOUR REPORTS WHEREVER YOU CAN
More time?
• Example of how to make a control chart using Excel.
• Calculating P-values to compare rates
• You can download this file at:http://innovationsforglobalhealth.com/files/Stat_Tests_v13.xls
Questions?
Thanks!