Safety, Can You Paradigm?

A Statistical Lament

Janet Turk WittesStatistics Collaborative

2

Harms identified late

•fenflurmine-phentermine (Fen Phen)

•Rofecoxib (Vioxx)

•Troglitazone (Rezulin)

•HRT (Premarin and PremPro)

•Celecoxib (Celebrex)

•Telithromycin (Ketek)

•Rosiglitazone (Avandia)

•Antidepressants, anti-epileptics….

3

Could we have identified these harms earlier?

•Troglitazone (Rezulin) -removed from market in 2000

Lots of liver abnormalities

Severe toxicities noted in 1997

Other equally effective drugs didn’t have same problems

4

Could we have identified these harms earlier?

•Troglitazone (Rezulin) -removed from market in 2000

• Rofecoxib (Vioxx) -removed from market in 2004

Every study showed excess heart attack

Attributed to benefit of naproxen

5

Could we have identified these harms earlier?

•Troglitazone (Rezulin) – removed from market in 2000

•Rofecoxib (Vioxx) -removed from market in 2004

•HRT (Premarin/PremPro)-major label change 2006

Heart attacks in Puerto Rican girls on oral contraception -1960’s

Men on estrogens had higher event rates – 1970’s

6

Could we have identified these harms earlier?

•Troglitazone (Rezulin) – removed from market in 2000

•Rofecoxib (Vioxx) -removed from market in 2004

•HRT (Premarin and PremPro)-label change 2006

•Celecoxib (Celebrex) – paper published 2005

•Telithromycin (Ketek) – major label change 2007

7

“CELEBRATE :: CELEBREX”

December 2004

8

How we statisticians help to save drugs

•We find safety boring

9

For efficacy we think hard about…

Outcomes

Population to study

Protocol

Analysis of primary outcome

Control of Type I error rate

Other outcomes

Missing data

Sensitivity analyses

10

How we statisticians save drugs

•Because we find safety boring….

We don’t look at preclinical and early Phase data

We don’t ask about

•Chemistry

•Biology

•What PK/PD studies show

•Safety part of analysis plan is an afterthought

11

How the statistical -police protect drugs

•We test hypotheses

•Put events in correct body system

•Give precise definitions

•No data dredging

•Too many type 1 errors if we dredge

12

And we divide and…

•conquer

•obfuscate

13

e.g. Neuropathy

Event T C

Neuropathic pain 1 0

Neuropathy 1 0

Neuropathy NOS 5 2

Neuropathy peripheral 2 0

… 2 1

…

…

14

e.g. Neuropathy

Event T C

…

Parathesia 3 2

Parathesia NOS 4 0

Parathesia other 0 1

…

Peripheral motor neuropathy 6 0

Peripheral sensory neuropathy 3 2

15

True(ish) data from a coxib

C TCardiac disorders 42 46Respiratory 33 29Vascular disorders 7 9

16

True(ish) data from a coxib

C TCardiac disorders 42 46•Angina 2 2•Angina aggravated 0 2•Angina unstable 0 3•…•Cardiac arrest 0 1•Cardiac failure congest 2 0•Coronary artery disease 4 7•…•Myocardial infarction 5 10

17

True(ish) data from a coxib

Respiratory 33 29•Dyspnea 1 3

Vascular disorders 7 9•Cerebral infarction 0 1

•Pulmonary embolism 0 2

•TIA 2 0

18

If you combined…

No. of people with at least one serious thromboembolic event or evidence of heart failure

Placebo Coxib

16 27

19

Other ways to save drugs

Modified Daley’s Rule:

Censor early and often

20

e.g., Rofecoxib- short follow-up

21

Through 36 months

22

With denominators (Bresalier et al. NEJM 2005 352:1092) (And see Adam Boyd’s poster!)

23

Known or suspected adverse events

•Monitor them

•Look at events, their (near) synonyms, labs

Are they real?

Are they too frequent?

24

Hierarchical multiplicity

•Think of biology

•Order hierarchy by decreasing

Biological plausibility

Objectivity

•Look for monotone decreasing hazard ratio

25

Which dose of celecoxib do you want?

26

APC Study (Placebo vs high dose)

Outcome n------------------------------------------------CV death  6  +MI   19+Stroke  26+CHF 29+Angina 34+CV procedure 46-----------------------------------------------Other CV 62

27

Adenoma Prevention with Celecoxib (APC) Study

HRCV death  5.1   +MI   3.8  +Stroke  3.4  +CHF   3.2 +Angina 2.1 +CV procedure 1.7 -------------------------------------------------Other CV 1.1 

28

APC Study

CV death  5.1  ( 0.6, 43.2)    +MI   3.8  ( 1.3, 11.4)   +Stroke  3.4  ( 1.4,  8.3)  

+CHF  3.2  ( 1.4,  7.4)  +Angina 2.1  ( 1.0,  4.3)  +CV procedure 1.7  ( 1.0,  3.1) ------------------------------------------Other CV 1.1  ( 0.7,  1.8)

29

APC Study

CV death  5.1  ( 0.6, 43.2) 0.14   +MI   3.8  ( 1.3, 11.4)  0.015 +Stroke  3.4  ( 1.4,  8.3)  

0.007+CHF  3.2  ( 1.4,  7.4)  0.006+Angina 2.1  ( 1.0,  4.3)   0.05+CV procedure 1.7  ( 1.0,  3.1) 0.05-------------------------------------------------Other CV 1.1  ( 0.7,  1.8) 0.7

Solomon (2006). Circulation 114:1028

30

Unknown harms: usual approach

•Respond by

Agonizing

Checking informed consent document

Asking for more frequent looks

Asking for more thorough analyses

•Worry about falsely discovered harm

31

Sentinel events

•Identify

•Follow in the next patients

•Invent formal statistical methods

32

Single sentinel event

•Childhood vaccine

•30 day follow-up for serious adverse events

•1 death occurred

•DSMB: did the vaccine cause the death?

33

Women’s Health Initiative

•Early in the trial, DSMB noted:

Increase in stroke

Increase in pulmonary embolism

Increase in myocardial infarction

•Possible sentinel events

Myocardial infarction

The big meanies: stroke, PE, MI

34

Proposal

1. Identify sentinel event (or cluster or rate)

2. Monitor for subsequent occurrence(s)

Have reasonable power

Be statistically unbiased (exclude sentinel)

Type 1 error rate may be large (~0.2)

Lachenbruch, Wittes: 2007

35

Safety report sample –abnormal lab values

•Time A B Total •Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________ SCREENING 0 0 0 •RANDOM 0 0 0 •WEEK 2 0 0 0 •WEEK 3 0 0 0 •WEEK 4 0 0 0 •WEEK 5 0 0 0 •WEEK 6 0 0 0 •WEEK 7 0 0 0 •WEEK 8 0 0 0 •___________________________________________________________________

36

But wait! You also get:

• Time A B Total • Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • RANDOM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 2 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 3 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 4 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 5 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 6 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 7 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 8 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) •_______________________________________________________________

37

And 150 pages of where’s Waldo

• Time A B Total • Point [N= 150] [N= 148] [N= 298] •_______________________________________________________________• SCREENING 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • RANDOM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 2 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 3 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 4 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 5 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 6 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 7 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • WEEK 8 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) • EARLY TERM 0 (0.00 %) 0 (0.00 %) 0 (0.00 %)UNSCHEDULED 0 (0.00 %) 0 (0.00 %) 0 (0.00 %) •_______________________________________________________________

38

And if this isn’t enough…

•Change from baseline where missing is counted as zero (change in HR=64????)

•Values out of temporal order

•Lots and lots of decimal places

•P-values to 3 and 4 significant digits

•Etc., etc. etc.

39

We need to change our habits

•Current statistical approach

One variable at a time

Template applied to all studies

No wonder the docs don’t ask us to work with them!

•Simple change in attitude

Safety parameters aren’t separable

Focus first from biological insights and previous hints

Then scan the other variables

Then refocus

40

Conclusions

•Worry about multiplicity, but not too much

Listen to Joe Heyse’s talk this afternoon

•Beware the censor-happy protocol and analysis

•Don’t be too much the statistician

•But don’t forget randomness