Adapting AHRQ Patient Safety Indicators to QIO Data

Jocelyn Andrel, MSPHCharles P. Schade, MD, MPHPatricia Ruddick, RN, MSN

Outline of Presentation

What are AHRQ Patient Safety Indicators?How can you use QIO data to get them?What are their characteristics in one state?How can you share them with hospitals?What do one state’s hospitals think of

them?How do they relate to other evidence about

safety in a state’s hospitals?

AHRQ Patient Safety Indicators

What they areHow to compute them

AHRQ Patient Safety Indicators: Background

Early 1990sDeveloped by the Agency for Healthcare

Research and Quality (AHRQ) to measure the safety of hospital care using administrative inpatient discharge data. The Indicators screen for problems that patients experience as a result of exposure to the healthcare system.

Concept of PSIs

Based on conditions that clearly reflect medical error (foreign body left in)

Based on conditions that could reflect medical error (PE or DVT)

Not based on underlying comorbidities

Steps to determine PSIs

1. Define the concepts and the evaluation framework

2. Search the literature to identify potential PSIs

3. Develop a candidate list of PSIs

4. Review the PSIs

5. Evaluate the PSIs using empirical analysis

Limitations

Some events don’t show up in discharge data– Adverse drug reactions– Medical events– Psychiatric events.

Administrative data may not address finer detail Patient Safety Indicators should be used to prompt

investigation into areas where the hospital could potentially improve quality of care

PSIs

Accidental puncture or laceration Complications of Anesthesia Death in low mortality DRGs Decubitus Ulcer Failure to Rescue Foreign body left in during procedure Iatrogenic pneumothorax Postoperative hemorrhage or hematoma Postoperative hip fracture

PSIs continued

Postoperative physiologic and metabolic derangement Postoperative pulmonary embolism or DVT Postoperative respiratory failure Postoperative sepsis Postoperative wound dehiscence Selected infections due to medical care Transfusion reaction Plus 4 Obstetric measures not addressed here

Converting ISAT data

General Instructions from AHRQ– 1. The data must be in SAS– 2. You may have to recode specific data elements

to match what is used in the software.Fortunately, conversion of the ISAT file to

comport with the AHRQ input requirements is fairly simple

Conversion Elements

Creating/Formatting Variables– Age– Length of Stay– Create variables for the number of diagnoses and the

number of procedures– Set payor to the code for Medicare– Format Hospital codes, Race, Sex, Key, Hospital ID,

DRG, Admission Source, Admission Type – Rename Diagnosis and Procedure codes

Major Diagnostic Codes from the HSE Claims Lookup Table

And then…

The ISAT file is ready to be input into the AHRQ Patient Safety Indicator programs

AHRQ Patient Safety Indicators:Results in a Single State

Methods

Adapted standard output (psp3 table at hospital level) to a graphic display and comparative report

Generated histograms of hospital performance on each indicator for 2000-2002

Generalized code to run with any state’s data as input

Results: Distribution of Hospitals

Some indicators appeared normally distributed

Some were highly skewed, with outliersSome appeared bimodal

AHRQ Risk Adjusted PSI RateFailure to RescueWV Hospitals, 2002

0

5

10

15

20

Nu

mb

er

of

Ho

sp

ita

ls

0.0

to

25

.0

25

.0 t

o 7

5.0

75

.0 t

o 1

25

.0

12

5.0

to

17

5.0

17

5.0

to

22

5.0

22

5.0

to

27

5.0

27

5.0

to

32

5.0

32

5.0

to

37

5.0

37

5.0

to

42

5.0

42

5.0

to

47

5.0

47

5.0

or

mo

re

Rate per 1,000 cases

Death rate in dischargeswith potential complicationsof care, e.g., pneumonia, DVT/PE, sepsis, acute renal failure, shock/cardiac arrest, GI hemorrhage/acute ulcer.

Definition

AHRQ Risk Adjusted PSI RateSelected Infections Due To Medical CareWV Hospitals, 2002

0

5

10

15

20

Nu

mb

er

of

Ho

sp

ita

ls

0.0

to .

5

.5 t

o 1.

5

1.5

to 2

.5

2.5

to 3

.5

3.5

to 4

.5

4.5

to 5

.5

5.5

to 6

.5

6.5

to 7

.5

7.5

to 8

.5

8.5

to 9

.5

9.5

or m

ore

Rate per 1,000 cases

Discharges with ICD-9-CMcode of 999.3 or 996.62 in anysecondary diagnosis fieldexcluding immunocompro-mised and cancer

Definition

AHRQ Risk Adjusted PSI RatePost-Operative SepsisWV Hospitals, 2002

0

5

10

15

20

Nu

mb

er

of

Ho

sp

ita

ls

0.0

to

2.5

2.5

to

7.5

7.5

to

12

.5

12

.5 t

o 1

7.5

17

.5 t

o 2

2.5

22

.5 t

o 2

7.5

27

.5 t

o 3

2.5

32

.5 t

o 3

7.5

37

.5 t

o 4

2.5

42

.5 t

o 4

7.5

47

.5 o

r m

ore

Rate per 1,000 cases

Elective surgical dischargeswith ICD-9-CM code for sepsis in any secondarydiagnosis field excludingimmunocompromised andcancer

Definition

Results: Statewide Values Over 3 Years

We also used the following format for the tabular report to individual hospitals

Most indicators based on small numerators statewide and appeared to show statistical fluctuation from year to year

Failure to rescue declining?Postop sepsis and DVT/PE increasing?

AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2000Statewide

Number Rate* Hospital Percentiles*Num-erator

Denom-inator

Crude Adjusted 10th 50th 90th

Complications of Anesthesia 11 29,053 0.4 0.3 0 0 1.1Death in Low Mortality DRGs 52 13,278 3.9 1.9 0 0.099 5.2Decubitus Ulcer 1,670 57,736 28.9 25.0 4.3 24.2 49.0Failure to Rescue 741 4,656 159.1 176.2 54.6 168.5 240.7Foreign Body Left in During Proc 6 131,145 0.046 0.063 0.013 0.027 0.222Iatrogenic Pneumothorax 124 119,616 1.0 0.9 0 0.448 2.1Infection Due to Medical Care 230 110,892 2.1 1.7 0 0.897 2.8Postoperative Hip Fracture 52 21,083 2.5 1.8 0 0.724 10.0Postop Hemor or Hemat-No Prday 62 29,028 2.1 1.9 0 0.570 4.3Postop Physio Metabol Derangmnt 21 12,576 1.7 1.1 0 0.397 6.0Postop Respiratory Failure 62 9,278 6.7 2.6 0 1.3 9.3Postop PE or DVT-No Prday 330 28,870 11.4 8.7 0 4.2 14.9Postoperative Sepsis 47 4,946 9.5 7.9 0 5.0 21.5Postoperative Wound Dehiscence 20 5,399 3.7 2.4 0 0 8.5Accidental Puncture/Laceration 290 131,049 2.2 3.3 1.7 2.6 3.9

*Rates per 1,000 eligible cases

Indicator

AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2001Statewide

Number Rate* Hospital Percentiles*Num-erator

Denom-inator

Crude Adjusted 10th 50th 90th

Complications of Anesthesia 7 28,926 0.242 0.185 0 0 0.185Death in Low Mortality DRGs 47 13,657 3.4 1.6 0 0 5.5Decubitus Ulcer 1,552 56,299 27.6 24.0 9.7 22.0 42.8Failure to Rescue 692 5,209 132.8 151.7 76.5 160.1 232.2Foreign Body Left in During Proc 5 129,442 0.039 0.056 0.008 0.026 0.131Iatrogenic Pneumothorax 119 117,966 1.0 0.9 0.044 0.281 1.5Infection Due to Medical Care 271 109,233 2.5 2.1 0.308 1.4 3.2Postoperative Hip Fracture 42 20,674 2.0 1.3 0 0.670 7.0Postop Hemor or Hemat-No Prday 60 28,876 2.1 1.9 0 0.855 5.3Postop Physio Metabol Derangmnt 15 12,971 1.2 0.6 0 0.138 1.3Postop Respiratory Failure 82 9,542 8.6 4.2 0 0 13.2Postop PE or DVT-No Prday 360 28,689 12.5 10.1 0 8.8 17.8Postoperative Sepsis 48 4,990 9.6 7.8 0 6.6 23.3Postoperative Wound Dehiscence 22 5,367 4.1 2.9 0 0 9.0Accidental Puncture/Laceration 272 129,303 2.1 3.2 1.5 2.8 4.0

*Rates per 1,000 eligible cases

Indicator

AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2002Statewide

Number Rate* Hospital Percentiles*Num-erator

Denom-inator

Crude Adjusted 10th 50th 90th

Complications of Anesthesia 11 26,188 0.420 0.347 0 0 0.347Death in Low Mortality DRGs 59 13,174 4.5 2.6 0 1.4 8.5Decubitus Ulcer 1,604 53,879 29.8 26.1 4.3 25.9 36.5Failure to Rescue 679 5,031 135.0 148.4 72.5 159.0 225.1Foreign Body Left in During Proc 4 124,902 0.032 0.050 0.008 0.026 0.050Iatrogenic Pneumothorax 100 114,155 0.9 0.8 0 0.639 1.4Infection Due to Medical Care 241 105,239 2.3 1.9 0.021 1.2 3.2Postoperative Hip Fracture 51 17,965 2.8 2.1 0 0 6.9Postop Hemor or Hemat-No Prday 64 26,155 2.4 2.2 0 0.610 3.8Postop Physio Metabol Derangmnt 29 11,993 2.4 1.8 0 0.147 3.1Postop Respiratory Failure 82 9,564 8.6 4.4 0 1.5 27.3Postop PE or DVT-No Prday 381 25,961 14.7 11.4 0 8.7 21.4Postoperative Sepsis 57 4,689 12.2 10.3 0 5.7 36.0Postoperative Wound Dehiscence 17 5,091 3.3 2.3 0 0 14.4Accidental Puncture/Laceration 219 124,798 1.8 3.0 1.5 2.6 3.7

*Rates per 1,000 eligible cases

Indicator

Report to Hospitals

Calendar year 2002, with offer of other years’ results

Tabular (see previous) and graphical formatExplanatory letter, definitions of indicatorsMailed to hospital patient safety contact or

HCQIP contactAsked for feedback on report contents and

utility

AHRQ Patient Safety Indicators--WV, 2002 (Risk Adjusted)Hospital A

Medicare Fee For Service Patients

0.001 0.01 0.1 1 10 100 1000

Indicator Rate (Per Thousand Cases)

(0.544)Complications of Anesthesia

(2.7)Death in Low Mortality DRGs

(25.7)Decubitus Ulcer

(160.6)Failure to Rescue

(0.018)Foreign Body Left in During Proc

(0.859)Iatrogenic Pneumothorax

(1.5)Infection Due to Medical Care

(3.8)Postoperative Hip Fracture

(2.3)Postop Hemor or Hemat-No Prday

(1.1)Postop Physio Metabol Derangmnt

(17.4)Postop Respiratory Failure

(10.0)Postop PE or DVT-No Prday

(16.4)Postoperative Sepsis

(0)Postoperative Wound Dehiscence

(2.1)Accidental Puncture/Laceration

State Range (10th-90th Percentile)

State Median (50th Percentile)

Hospital Risk Adjusted Value

Patient Safety Indicators:

Implications for WVMI’s Patient Safety Project

Specific Goals of the WV Patient Safety Project

Establish a system of confidential reporting for medical errors and near misses

Stimulate reporting of such events by developing a non-punitive response system

Provide feedback of surveillance data at appropriate levels of aggregation

Educate consumers of healthcare about patient safety guidelines

Comparing PSI Data to the Patient Safety Data Purpose:1. Ascertain the usefulness of the PSI

data in hospitals in West Virginia2. Compare the data received from the

PSI data to the data received from the Patient Safety Project

3. Explore further opportunities for quality improvement projects

PSI/Patient Safety Data Study

CEOs and Quality Improvement staff from 41 acute care West Virginia hospitals received:

1. Information letter2. Patient Safety Indicator definitions3. Table which showed the actual number of specific

incidences of each PSI (2002), crude and adjusted rates, and comparative percentiles of all hospitals in the state combined

4. Graphical representation of the data presented in the table

5. Brief questionnaire on the usefulness of the graph and tables

Patient Safety Questionnaire

Feedback on the Patient Safety Indicator Reports

 Please take a minute or two to tell us your reaction to the enclosed reports. Your responses will be kept confidential and used only for evaluating this project.

1. Please check the box that most closely describes your role in the hospital

Quality improvement staff

Patient safety staff

Medical staff

Clinical nursing staff

Administration

Other ________________ 

2. Please circle the number indicating the extent to which you agree or disagree with each statement, where:

5 = strongly agree4 = agree3 = indifferent2 = disagree1 = strongly disagreeIf a question is not applicable to your situation, please leave

it blank. Strongly agree...strongly disagree a. The patient safety indicator reports were easy to understand

5 4 3 2 1 b. The graphic report was easier to use than the tabular report

5 4 3 2 1 

Patient Safety Questionnaire, cont.

c. The tabular report provided more information than the graphic report

5 4 3 2 1 d. My hospital’s indicator results, compared with the state’s rates,

are about what I would have expected5 4 3 2 1

 e. I want to share the report with colleagues in my hospital

5 4 3 2 1 f. I need additional information about one or more of the

indicators 5 4 3 2 1

Patient Safety Questionnaire, cont.

3. Please tell us anything you liked about the reports:

   

4. Please let us know of anything you did not like about the reports:

  

  5. Finally, please let us know any questions you’d like answered about the reports:

Patient Safety Questionnaire, cont.

Results(14/41 questionnaires returned)

5 4 3 2 1a. The PSI reports were easy to understand 28% 42% 14% 14%

b. The graphic report was easier to use than the tabular report

35% 28% 14% 14% 7%

c. The tabular report provided more information than the graphic report

21% 28% 35% 7% 14%

d. My hospital’s indicator results, compared with the state’s rates, are about what I expected

25% 25% 33% 17%

e. I want to share the report with colleagues in my hospital

35% 28% 21% 7% 7%

f. I need additional information about one or more of the indicators

50% 7% 14% 21% 7%

Likes/dislikes about the PSI Reports

Likes: Good overview of our results Serves as a step for further analysis Great idea-shared this with Department of

Medicine Graphs were self-explanatoryDislikes: Leaves many questions unanswered Need more current information Would like to set up and run on their own

ExamplesPatient Safety Indicator Patient Safety Event

Complications of anesthesia Adverse Clinical Event

-sedation management

-complication/monitoring

Decubitus ulcers Adverse Clinical Event

-skin integrity

-decubitus

Foreign body left in during procedure

Adverse Clinical Event

-operative/invasive procedure

-instrument/needle/sponge count

Infection due to medical care Adverse Clinical Event

-infection

May have to search several fields to find

coordinating PSI

Conclusions Data captured from PSIs may best be used to investigate

potential patient safety problems when hospitals compare PSIs to the coordinating medical error on the incident reporting tool since:

Some events don’t show up in discharge data that are captured in the incident reporting tool, e.g.

– Adverse drug reactions– Administrative events – Fall events – Employee events– Visitor events

PSI data is more general and may have to use several fields in the incident reporting tool to capture complete PSI data

WVMI plans to:

Compare 2003 PSI data with data from Web-based incident reporting tool for hospitals that are part of the WV Patient Safety Project

– Unable to compare 2002 PSI data with patient safety data since the hospitals participating in the Patient Safety Project did not start until middle of 2002; and indicators do not correspond exactly.

– Provide this information to each participating hospital in order that they will be able to compare their reporting rates to PSI data

Source

AHRQ patient safety indicator programs http://www.qualityindicators.ahrq.gov

Conversion routines and hospital output code:

– jandrel@wvmi.org– cschade@wvmi.org

WVMI’s Patient Safety Project– pruddick@wvmi.org