Post on 17-Jan-2020
Analysis of the impact of
AHRQ exclusions on the
variation in Patient Safety
Indicator (PSI) values
by Vladimir Stevanovic
Patient Safety Subgroup meeting
Paris, 26 May 2011
Background HCQI Expert Group meeting in June 2009
• Concerns were raised that PSI data may reflect rather
coding and registration practices than actual differences
in patient safety
• Several countries expressed reservations about the
publication of PSIs in Health at a Glance 2009 due to
perceived risk of misinterpretation
• The Secretariat proposed to undertake further analysis
with the aim of validating the methodological approach
Background PSI Subgroup meeting in October 2009
• The Secretariat presented a preliminary analysis on
the impact of AHRQ exclusions by using NZ data
• The findings implied that the exclusions have
varying impact on the country results and may distort
indicator values apart from the obstetric ones
• The Secretariat proposed further analysis to be
undertaken through a voluntary subsample of countries
by replicating the same methods and using UPIs
Background HCQI Expert Group meeting in June 2010
• Ten countries participated in the replication analysis:
CAN, DNK, FIN, ISR, NOR, SNG, SPA, SWE, SWI, USA
• The results confirmed the previous findings that
differences in coding practice, admission type definition
and long LOS exclusion are likely to be the most
important driving factor behind variations
• The decision was made to collect additional data by
the means of regular HCQI 2010/11 data collection
in order to better understand the impact of exclusions
Objectives
To improve international comparability of PSIs by:
• Assessing the impact of admission type (ADMT) and
and length of stay (LOS) exclusions on the rates for - PSI 07 Catheter-related bloodstream infection,
- PSI 12 Pulmonary embolism or deep vein thrombosis,
- PSI 13 Postoperative sepsis
• Exploring whether these exclusions account for any
undesired or increased variation across countries
The scope does NOT include exclusions that are
inherent to the concept of an indicator by their nature
Methods
Catheter-Related Bloodstream Infection
Age =15 y or >15 y?
Case is
assigned to MDC 14
or the PDx is listed
in table M3?
Exclude
PDx is immunocompromised
state (list C-1 / W-1), or cancer (list C-2/W-2)?
SDx is immunocompromised
state (list C-1 / W-1), or cancer (list C-2/W-2)?
yes Exclude
Excludeyes
PDx is
identical to the
numerator definition?
no
Add case to
denominator
population
Add case to
numerator
population
SDx is
identical to the
numerator definition?
no no
Calculate mean
number of secondary
diagnoses and ALOS of
denominator population
and report (6)
Count and report
denominator
cases (7)
Count numerator cases
and report (8)
no
no
yes
All hospital discharges
of one year
yes Exclude
yes
Catheter-related bloodstram infections – pre-exclusions
All hospital discharges of one year,
age =15 years or > 15 years and
LOS < 24 hours or one day
Count and report number of
cases / admissions, ALOS, and
mean number of secondary
diagnoses (1)
PDx is
identical to the
numerator definition?
yes
Count and report number of
patients (countries with UPI
only) / discharges (2)
no
End
Catheter-related bloodstram infections – impact of length of stay (LOS) exclusions
Count numerator cases and
report (5)
LOS
is < 48 hours or
< 2 days?
yes
LOS
is < 24 hours or
< 1 day?
yes
Count denominator
cases and report (4)
Count denominator
cases and report (5)
SDx is
identical to the
numerator definition?
yes
SDx is
identical to the
numerator definition?
Count numerator cases and
report (4)
yes
yes
yes
LOS > or = 48
hours or 2 days?
yes no
Perform pre-exclusion calculations,
see seperate flowchart
Count and report
denominator cases (3)
SDx is
identical to the
numerator definition?
Count and report
numerator cases (3)yes
Pre-exclusion and
Post-exclusion stats (all discharges in a year AND
age>=15 years AND LOS >= 24h)
- the total number of discharges
- the average length of stay
- the average number of SDX
Length of stay and
Admission type exclusions
- the numerator data
- the denominator data
Additional data
Countries
• Australia
• Belgium
• Canada
• Denmark
• Finland
• France
• Iceland
• Israel
• Italy
• New Zealand
• Singapore
• Spain
• Sweden
• Switzerland
• United States
Results PSI 07 Catheter-related bloodstream infection (LOS excl)
Results PSI 07 Catheter-related bloodstream infection (LOS excl)
Spearman-rank test (no LOS excl vs <1 day, <2 days) = 0.89, 0.84 (both p<0.01)
Results PSI 12 Postoperative PE or DVT (LOS excl)
Results PSI 12 Postoperative PE or DTV (LOS excl)
Spearman-rank test (no LOS excl vs <1 day, <2 days) = 0.78,(p<0.01), 0.59 (p=0.07)
Results PSI 13 Postoperative sepsis (LOS excl)
Results PSI 12 Postoperative sepsis (LOS excl)
Spearman-rank test (no LOS excl vs <1-3 days, <4 days) = 0.93-97, 0.78
Results PSI 12 Postoperative sepsis (LOS excl)
Results PSI 13 Postoperative sepsis (LOS & ADMT excl)
Results PSI 12 Postoperative sepsis (LOS & ADMT excl)
Spearman-rank test (no ADMT vs with ADMT) = -0.19 (p=0.60)
Results PSI 12 Postoperative sepsis (ADMT distribution)
Results PSI 12 Postoperative sepsis (modified excl criteria)
Results PSI 12 Postoperative sepsis (modified excl criteria)
Publication PSI 07 Catheter-related bloodstream infection
• Min/Max ratio = 100 fold
• Variation in coding practices?
• Inflammatory conditions?
Recommendations
1. The exclusion criteria for PSI 07 Catheter-related
bloodstream infections and PSI 12 Postoperative
pulmonary embolism and deep vein thrombosis are
appropriate, hence there is no need to change the
existing LOS<2 days exclusion.
2. For PSI 13 Postoperative sepsis, exclusions of
discharges with LOS<4 days and non-elective (acute)
type admissions account for an increased variation
between countries and cause bias in the resulting
postoperative sepsis rates. It is recommended to
drop the admission type exclusion criterion from
the algorithm and to use the modified LOS<3 days
exclusion instead.
Recommendations - cont.
3. For PSI 07 Catheter-related bloodstream infection,
remaining ambiguities in the definition make this
indicator not fit for reporting at this moment.
4. The following indicators seem to be robust enough and
are therefore recommended for publication in
Health at a Glance 2011:
- PSI 05 Foreign body left in during procedure
- PSI 12 Post-operative pulm. embolism or deep vein thrombosis
- PSI 13 Post-operative sepsis
- PSI 15 Accidental puncture or laceration
- PSI 18 Obstetric trauma - vaginal delivery with instrument
- PSI 19 Obstetric trauma - vaginal delivery without instrument
Optimal system requirements
• Unique patient identifier (UPI)
• Near miss and adverse events register
• Present on admission (POA) flag
• Standardised registration/coding practice
• Mapping between classification systems
• Calculation method/algorithm adjusted for
international comparison - exclusions inherent to the concept of PSI
- adjustments for the effect of confounders
• Comment on findings from the analysis,
• Decide on whether to change the algorithm used for
the calculation of PSI13 Postoperative sepsis as
recommended by this report,
• Decide on whether to adjust patient safety indicator
rates by the mean number of secondary diagnoses
among patients at risk as (previously presented by Saskia Drösler
and Patrick Romano and discussed in 2009/10),
• Make recommendations for the continuation of the
development work on PSIs (agenda item 4).
• Make recommendations on which patient safety
indicators should be published in the chapter on
Quality of Care in Health at a Glance 2011 (agenda item 5).
Members of the Patient Safety Subgroup are invited to