Validation of Acute Stroke in Medicare Data against WHI

Kamakshi Lakshminarayan, MD, PhD presented by Dale Burwen, MD, MPH

WHI Investigators Meeting

May 3-4, 2012

Preliminary results; do not distribute

1

Writing Group

• Kamakshi Lakshminarayan • Dale Burwen• Joe Larson• Beth Virnig• Wolfgang Winkelmayer• Norrina Allen• Monica Safford• Marian Limacher

2

Background

• Starting in 2010, stroke outcomes in WHI will be adjudicated in only a quarter of participants

• Medicare data provide potential for expanding outcome ascertainment

• Little is known about validity of Medicare claims for ascertaining neurologist adjudicated strokes

3

Objective

Compare agreement between various algorithms to detect stroke hospitalizations in Medicare claims data and neurologist adjudicated stroke outcomes in WHI

4

Methods – Study Population

Inclusions

• Observational study women• With Medicare Parts A&B,

Fee-For-Service at the time of WHI enrollment 1993-1998N=27,739

• Those who age into Medicare Parts A&B, Fee-For-Service after enrollment until 2007 N=21138

• Total N = 48,877

Exclusions

• Managed care at the time of their WHI enrollment

• Participants are censored as they enter into managed care

• Those with WHI adjudicated stroke outcomes prior to CMS eligibility are excluded

• Participants are censored 7 days after WHI stroke

5

Methods• Randomly split into training & test data sets• Training set N = 24,432 • Test set N = 24,495• Analysis to date confined to training set

6

Stroke in WHI

• Rapid onset of persistent neurologic deficit attributed to obstruction or rupture of brain arterial system.

• Deficit is not known to be secondary to brain trauma, tumor, infection, or other cause.

• Deficit must last > 24 hours unless death supervenes or there is a lesion compatible with an acute stroke on CT or MRI.

7

Defining stroke in Medicare

Used 1993-2007 hospital data (MedPAR file)

8

Definition ICD-9 code Code Position

1. All stroke 430, 431, 433.x1, 434.x1, 436, 437.1x, 437.9x

Any position

2. Primary position stroke Same as above Primary position

3. Ischemic stroke 433.x1, 434.x1, 436, 437.1x, 437.9x

Any position

4. Hemorrhagic stroke 430, 431 Any position

Results pertain to the 1st definition

Events Included in Analysis

Universe of events:– WHI confirmed strokes after neurologist

adjudication– All hospitalization claims from Medicare data

(stroke and non-stroke)– Goal is to classify each claim into stroke vs. not

• Definition of matched events: – WHI stroke & CMS stroke +/- 7 days

• Sensitivity analysis with wider intervals (14 days)

9

Match Results (Stroke in any diagnosis position)

WHI Yes WHI No

CMS Yes 478 374

CMS No 105 55995

10

Kappa 0.66

Reasons for Disagreement

WHI Yes, CMS No(N=105)

• Hospital claim found +/- 7days; but claim did not have diagnosis codes meeting stroke definition

• No hospital claim found +/- 7 days; outpatient stroke according to WHI

• No hospital claim found +/- 7 days

54% (n=57)

5% (n=5)

41% (n=43)

11

Reasons for Disagreement

WHI No, CMS Yes(N=374)

• Self-report of stroke or Transient Ischemic Attack (TIA), with hospitalization +/- 7 days – Adjudicated as TIA or carotid disease – Adjudicated as no outcome– Not adjudicated due to administrative

reasons

• Self-report of other hospitalization • No report of hospitalization

– (case ascertainment of WHI)

24% (n=89)

7% (n=28)

13% (n=50)

3% (n=11)

21% (n=78)

55% (n=207)

12

Original vs. Modified Analysis

WHI Yes WHI No

CMS Yes 478 374

CMS No 105 55995

13

WHI Yes WHI No WHI No Hospitalization ReportedOr Administrative denials

CMS Yes 478 156 218

CMS No 100 55995

Outpatient strokes 5

Validation Performance

Original Analysis

• Sensitivity: 82.0%• Specificity: 99.3% • PPV: 56.1% • Kappa: 0.66

Modified Analysis

• Sensitivity: 82.7%• Specificity: 99.7%• PPV: 75.4%• Kappa: 0.79

14

Validation PerformancePrimary Position

Original Analysis

• Sensitivity: 73.9%• Specificity: 99.6% • PPV: 63.9% • kappa: 0.68

Modified Analysis

• Sensitivity: 74.6%• Specificity: 99.8%• PPV: 82.3%• kappa: 0.78

15

Discussion• Initial WHI vs. Medicare agreement was moderate• Key reason for a CMS event without WHI match was

lack of WHI report of hospitalization – Possible reasons: Inadequate recall; disability/death

and lack of proxy report• Limiting analysis to CMS events that could be

evaluated with WHI medical records increased PPV to 75% – Primary position diagnostic codes PPV = 82%

• False positives due to TIA were in a minority; mainly other diagnosis

16

Discussion (cont.)• A key reason for WHI stroke without matching CMS

stroke was that WHI picked up a lot of strokes coded with a variety of other diagnosis codes. – However, there was no predominant code to suggest how

to modify our algorithm

• Another important reason was lack of CMS hospital claim within the selected time interval (+/- 7 days)– Wider time interval picked up a minority

17

Next Steps• Further exploration of reasons for

disagreement • Test additional algorithms • Consider incorporation of Medicare

procedures/diagnoses for rehabilitation– To increase specificity for stroke vs. TIA, and

current vs. historical stroke

18

Thank you!

Questions?

19