2006 FDA/Industry Workshop Advantages and Challenges of Bayesian Clinical Trials Increased Bayesian...

2006 FDA/Industry WorkshopAdvantages and Challenges of Bayesian Clinical Trials

Increased Bayesian Submissions: A future state of Drug applications?

Stacy R. Lindborg, Eli Lilly & Co.September 29, 2006

FDA/Industry Workshop 2006S.Lindborg

Company ConfidentialCopyright © 2000 Eli Lilly and Company

2

Acknowledgments

Melissa E. Spann, Eli Lilly & Co.

John W. Seaman, Jr., Baylor University



3

Net Cost: $802 Million Invested Over 15 Years

5,000–10,000Screened

250Enter Preclinical

Testing

5Enter

Clinical Testing

1

Compound Success Rates by Stage

16

14

12

10

8

6

4

2

0

Phase II100–300 Patient Volunteers Used to Look for Efficacy and Side Effects

Phase III1,000–5,000 Patient Volunteers

Used to Monitor AdverseReactions to Long-Term Use

FDA Review ApprovalAdditional Post-

Marketing Testing

Phase I 20–80 Healthy Volunteers Used to

Determine Safety and Dosage

Preclinical TestingLaboratory and Animal Testing

Discovery(2–10 Years)

Years

New Product Development –A Risky and Expensive Proposition

Source: Tufts Center for the Study of Drug Development

Approved by the FDA



4

Investment in Research and Development Continues to Grow

Source: PhRMA 2006 Industry profile, (http://www.phrma.org/files/2006%20Industry%20Profile.pdf)



5



6

Challenges for the Implementation of Bayesian methods

Lack of Bayesian analyses in Regulatory interactions– Many reasons (e.g., computing, FDA acceptance?)

– Focus on classical statistics by many graduate programs resulting in an unfamiliarity with Bayesian inference among many statisticians (PhRMA & regulatory globally)

– Uninformed scientists (even statisticians) about how a Bayesian analysis is performed (i.e. prior elicitation).

– Incorrect Conclusion: Bayesian inference is not acceptablewithin the pharmaceutical industry



7

2006 ENAR Roundtable

Topic: The Use of Bayesian Inference in Drug Approval

Un-informed comment from a (senior) colleague in the Pharmaceutical industry …

“Bayesian inference can never be used in credible settings. If you don’t like the answer then one can simply change the prior to get the answer you like!”

The basic tenants of a good trial are the same for both Bayesian & frequentist trials…Bayesian trials start with a sound clinical design (source: FDA CDRH draft Guidance)



8

FDA/Johns Hopkins WorkshopMay 20-21, 2004

Can Bayesian approaches to studying new treatments improve regulatory decision-making?”

Senior FDA leadership: Lester Crawford, Janet Woodcock, Bob Temple, Center Division Directors (CVM, CDER, CBER, CDRH)

Web-cast of meeting: www.prous.com/bayesian2004

Website to speaker slides: http://www.cfsan.fda.gov/~frf/bayesdl.html

Clinical Trials (special issue, dedicated to discussion): Vol 2, No 4, Aug 2005



9

FDA/Johns Hopkins WorkshopA few points…

Use of Bayesian methods: if you use these models, “all will get easier” and drug development will speed up

– Temple: “This statement is not true in my opinion. In some cases my prior will be very negative, as in case of drugs for sepsis…” (don’t assume it will always speed up)

– T.Louis: Sometimes ‘better’ might require more patients to overwhelm the prior

– G.Campbell: It’s a lot of work for everyone

– J.Seigel: Makes it harder to approve drugs that aren’t already a foregone conclusion. Also makes it easier to approve drugs we already know work.

– Pfizer did 1000’s of simulations to understand operating characteristics. What sample size is needed to accomplish traditional power estimates?

Simulations need to plays important role – operating characteristics of posterior distribution (i.e., frequentist characteristics) are necessary (D.Rubin)

– Operating characteristics: Type I error or some analog protects the public (G.Campbell)

– Don’t just do where we have to. They make sense (S.Ellenberg)

The issue of transparency is very important (S.Ellenberg)



10

Critical Path is the FDA's premier initiative to identify and prioritize the most pressing medical product development problems and the greatest opportunities for rapid improvement in public health benefits. Its primary purpose is to ensure that basic scientific discoveries translate more rapidly into new and better medical treatments by creating new tools to find answers about how the safety and effectiveness of new medical products can be demonstrated in faster timeframes with more certainty, at lower costs, and with better information.

FDA Critical Path Initiative



11

Critical Path Topic 2: Streamlining Clinical Trials

Creating Innovative & Efficient Clinical Trials & Improved Clinical Endpoints

Advancing Innovative Trial Designs

• Design of Active Controlled Trials• Enrichment Designs• Use of Prior Experience or Accumulated Info. in Trial Design• Development of Best Practices for Handling Missing Data• Development of Trial Protocols for Specific Therapeutic Areas• Analysis of Multiple Endpoints



12

Bayesian Methods in Clinical Research?A few points worth considering…

• Ability to calculate probability of interest and interpret easily– No arguments that p-values indirectly aide in decision making– P-value assumes H0 is true, can’t represent if H0 is true.– Bob Temple: “using p=.05 is not an appropriate measure of strength of information.

Everyone knows p-values are ‘stupid’”

• Joint probability estimation on 2+ parameters (multi-level structure & compute joint posterior distribution for all unknowns)

– Critical Success Factors

• Adaptive designs– flexible, no penalty for looking. Learn faster

• Borrow Strength (historical data, trials, across pt. groups & Dx)

• FDA Modernization Act of 1997 - least burdensome means of demonstrating effectiveness or substantial equivalence



13

Bayesian Methods in Clinical Research?• Non-inferiority trials lend themselves to Bayesian analyses

– Ease of incorporating variability appropriately

• Ability to predict - posterior predictive distributions. The ultimate goal of clinical trials is to predict an outcome variable of interest and to compare predictions across treatments.

Other areas for predictive probabilities: e.g., pretrial predictions – used to compare different sample sizes & designs and aid to judge competing trials considering different treatments. e.g.2: wrt interim analyses, given the data what is the chance of obtaining a ‘significant’ result?

• Ease in complex modelingAppropriately account for variability (e.g., different correlations over course of trial in key measures, between center differences, appropriate model for meta-analysis – including the case where the control rate varies across trials…)



14

NEW Draft CDRH Guidance

Draft Guidance available: http://www.fda.gov/cdrh/guidance.html



15

Where do we go from here?

B. Temple – apply Bayesian and frequentist methods to a classical study design; compare.

– Consider retrospective studies to accomplish these types of comparisons and develop confidence.

Olanzapine Intramuscular: phase III study in clinically agitated

patients with Schizophrenia



16

Phase III Study DesignPhase III Study DesignClinically agitated patients with schizophreniaClinically agitated patients with schizophrenia

Screening Period Double-Blind Therapy Period

(Screen) (Day 1)

24 hrs*

Randomisation and1st injection

1 day

(Day 4)

2 hrs

(Day 2) (Day 3)

<1 day 1 day

Eligible

1 day

Visit 2 Visit 3 Visit 4 Visit 5 Visit 6Visit 1

IM placebo

IM olanzapine(10 mg)

oral olanzapine (5-20 mg/day)

Patients

IM haloperidol(7.5 mg) oral haloperidol (5-20 mg/day)

*1 to 3 inj withinthe first 20 hrs

Antipsychoticsstopped. No

benzodiazapinesfor 4 hrs prior to

1st injection.

(Day 5)

Primary Objectives (IM):2 hr change, PANSS-EC

1. Superiority of Olz to Pla2. Non-inferiority of Olz to Hal

Secondary Objectives:

Safety of Olz relative to Pla and Hal



17

Why Bayesian Adaptive Design (AD)?

Among other reasons, a Bayesian AD provides an ability to:

• Formally leverage prior data

• Ability to model key efficacy and safety variables and continuously monitor and report updated predictive probabilities of interest

Bayesian ANOVA: same data model as employed in frequentist analysis

( , )tr geoijk j ky N



18

Olanzapine RAIM Adaptive Design benefits?

Appropriately employed (the where & how):• Decrease cycle time

– Greater than expected effect size (vs. Hal & Pla), stop sooner?

• Improve p(TS)– Increase quantity and precision of information

• Ethics– Optimize patient treatment– Minimize patient exposure to ineffective doses/Tx

Minimize Hal exposures decreased dystonia reactions? Minimize Placebo exposures?

Additional Olz IM data?



19

Prior data exploredOral vs. IM PK data? IM & oral PK profiles very similar, efficacy timing very different

Efficacy Scales from Phase II? No PANSS EC data Tranquallization scale (e.g., ACES)

Conflicting results between 2 trials Early Tranquallization scales failed to discriminate between Agitation and Calming

Scale: 1 = Alert (including agitated, aggressive), 2 = Tranquillization (calming of mood or motor activity)

Efficacy data from Olanzapine Oral? HGAJ data, PANSS EC inclusion entry criteria imposed – provided estimate of

treatment difference with Agitation (not very precise!) Responder analyses from Olanzapine and Haloperidol

Safety data: Olanzapine Oral & Haloperidol literature? A lot of data for prior use



20

Informative (A) Unfavorable (A) Diffuse (B) Pessimistic (A) 1 ~ N( 8, 1/36) j ~ N(j

tr) j ~ Beta()[ 35,21] 1 ~ N( 8, 1/36) 2 ~ N( 8, 1/36) (j

tr) 1/2 ~ U(, Btr) () 1/2 ~ U(, B) 2 ~ N(, 1/36) 3 ~ N( 5, 1/36) () 1/2 ~ U(, B) 3 ~ N( , 1/36) () 1/2 ~ U(, B) () 1/2 ~ U(, B)

Prior Distributions:Mean Change in PANSS-EC

20 0 200

0.05

.075

2.478 107

dnorm x 8 6( )

dnorm x 8 6( )

dnorm x 5 6( )

2135 x

0.075

-35 21

0

trj

3tr

2tr

1tr

20 0 200

0.05

.075

2.478 107

dnorm x 8 6( )

dnorm x 8 6( )

dnorm x 5 6( )

2135 x

0.075

-35 21

0

trj

3tr

2tr

1tr

1=IMHal7.5 2=IMOlz10 3=IMPla

*Center all distributions at zero to reflect a prior belief that all treatments are ineffective

**Assume a different data model with flat prior distributions across the support of the parameter

***Reflect potential lack of confidence in IMOlz10 relative to the IMPla and IMHal7.5

* ** ***

Informative prior elicitation was performed by way of a planning committee comprised of statisticians, physicians, pharmacokinetic scientists, and regulatory scientists.



21

Other prior distributions elicited

• Probability of Dystonic Reaction (Safety)• Probability of clinical response without Dystonia


(See back-up slides for details)



22

Bayesian Adaptive Non-Inferiority Design with Safety Assessment

Fixed Allocation

Calculate PosteriorProbabilitiesSafety

STOP

Efficacy

STOP

Futility

STOP

MinimumSample

Sizes

STOP

MaximumSample

Sizes

AdaptiveAllocation

CONTINUE

RE

PE

AT

PR

OC

ED

UR

E

HalOlzPla

HalOlz

OlzPla

Select Treatment

Treat PatientUpdate Data



23

Adaptation and Stopping

Minimum sample size required before adaptation begins including stopping for futility or efficacy: 30 placebo, 60 Hal IM and 60 Olanzapine IM

• Stopping Criteria: Stop for Futility: P(Non-Inferiority)<0.20 or P(Superiority)<0.20Stop for Efficacy: P(Non-Inferiority)>0.90 and P(Superiority)>0.90

• Maximum sample size allowed (# observed in trial given retrospective nature)

• Stop for Safety



24

MCMC Multiple Integration Estimation

• Gibbs sampling used to estimate the marginal distribution.

• Convergence for reliable parameter estimation was assessed by visual inspection of graphical diagnostics and the calculation of the Gelman-Rubin (Gelman 2004) convergence diagnostic

Computing: SAS IML WinBugs



25

Frequentist vs. Bayesian Conclusion

p<.001

Lower bound of 1-sided

97.5% confidence interval 0 but >LL

Frequentist

Olz Superior to Pla

Olz Noninferior to Hal

3.05

Lower Limit

IMHal7.5 Better IMOlz10 Better OlzHal

-1.121-sided 97.5% CI

Non-inferiority margin between Olz and Hal, 40%

i.e., max. allowable difference in pt response



26

Bayesian Approach to Non-Inferiority and Superiority

~ ,trj jy N

~ 0,tr trj N

1=Hal 2=Olz 3=Pla

1 2~ 0,tr trUniform B

1 2~ 0,Uniform B

• Probability Olz is non-inferior to Hal

• Probability Olz is superior to Pla

2 10.40 0 |P y

2 3 0 |P y

j is the change in PANSS-EC 2 hrs post baseline



27

Frequentist vs. Bayesian Conclusion

p<.001

posterior prob = 1

Lower bound of 1-sided

97.5% confidence interval 0 but >LL

posterior prob =1

Frequentist Bayesian

Olz Superior to Pla

Olz Noninferior to Hal

The probabilities of superiority of Olz vs. Pla and non-inferiority of Olz vs Hal were 1 for all priors



28

The true benefit of retrospective design Sample Sizes Real Trial Retrospective Adapt. Dx Haloperidol IM 126 60 Olanzapine IM 131 66* Placebo IM 54 30

24 patients could have been spared Placebo treatmentFewer Haloperidol exposures (i.e., decreased dystonic reactions)

$ 1 mil Savings ($6.4k/pt, clinical grant savings)40% time savings (stop after 6 of 10 mo.)

Trial stopped: criteria met to declare Olanzapine IM superior to Placebo and non-inferior to Haloperidol IM*note: max value based on prior sensitivity analysis (min=62)



29

Future role of Bayesian inference in the pharmaceutical industry?



30

Back-up Slides



31

Convergence

•A separate experiment was conducted to assess convergence of prediction estimates.

•We generated n=100 data sets then used over dispersed initial starting values for m=3 chains to calculate the Gelman-Rubin convergence diagnostic.

•A burn in of 100 with 1000 iterations yielded GR values ranging from .999537 to 1.002757 indicating that the estimated predictions converge under diffuse priors.

•The same type of experiment was performed for the effect size model with similar results.



32

Priors Distributions: Probability of Dystonia

Informative Diffuse Pessimistic

1D ~ Beta(15.85, 85.16) 1

D ~ Beta(1, 1) 1D ~ Beta(15.85, 85.16)

2D ~ Beta(2.04, 52.04) 2

D ~ Beta(1, 1) 2D ~ Beta(15.85, 85.16)

3D ~ Beta(1.54, 54.30) 3

D ~ Beta(1, 1) 3D ~ Beta(1.54, 54.30)

† ** ***

0 0.20

10

20

3030

0

Gbeta D 15.85 85.16 0 1( )

Gbeta D 2.04 52.06 0 1( )

Gbeta D 1.54 54.30 0 1( )

.350 D

Dj

30

0.35 0

0

1D

2D

3D

0 0.20

10

20

3030

0

Gbeta D 15.85 85.16 0 1( )

Gbeta D 2.04 52.06 0 1( )

Gbeta D 1.54 54.30 0 1( )

.350 D

Dj

30

0.35 0

0

1D

2D

3D


† Diffuse priors for binomial model were used when unfavorable priors for mean change in patient condition were used






33

Priors for Probability of Patient Responding without Dystonia

Informative Diffuse Pessimistic

1R ~ Beta(8.80, 10.2) 1

R ~ Beta(1, 1) 1R ~ Beta(8.80, 10.2)

2R ~ Beta(4.50, 5.10) 2

R ~ Beta(1, 1) 2R ~ Beta(4.80, 8.40)

3R ~ Beta(4.80, 8.40) 3

R ~ Beta(1, 1) 3R ~ Beta(4.80, 8.40)

† ** ***

0 0.5 10

2

44

0

Gbeta Res 8.8 10.2 0 1( )

Gbeta Res 4.5 5.1 0 1( )

Gbeta Res 4.8 8.4 0 1( )

10 Res

Rj

5

1 0 0

1R

2R

3R

0 0.5 10

2

44

0

Gbeta Res 8.8 10.2 0 1( )

Gbeta Res 4.5 5.1 0 1( )

Gbeta Res 4.8 8.4 0 1( )

10 Res

Rj

5

1 0 0

1R

2R

3R


† Diffuse priors for binomial model were used when unfavorable priors for mean change in patient condition were used






34

Joint Posterior Predictive Probability & Allocation Probabilities

The joint posterior predictive probability of the next patient responding to treatment and not experiencing dystonia is

The allocation probabilities are determined by

, | , , , , , , , 1R Dj j j jR R R S S S

j j j j j j j j j j R R R D D Dj j j j j j

R DP r s r n s n

n n

3

1

, | , , , , , , ,

, | , , , , , , ,

R R R S S Sj j j j j j j j j j j

jR R R S S S

j j j j j j j j j j jj

p r s r n s nP T

p r s r n s n



35

Predictive Probabilities:Responder & Safety

Rj j

R R Rj j j

R

n

1Dj j

D D Dj j j

D

n

The posterior predictive conditional probability that the next patient responds to treatment (40% reduction in agitation two hours post baseline) given that he does not experience dystonia

The posterior predictive probability that the next patient does not experience dystonia is

~ ,R Rj j jR Binomial n

~ ,R R Rj j jBeta

~ ,D Dj j jD Binomial n

~ ,D D Dj j jBeta

(See back-up slide for joint posterior predictive probability computations & allocation probabilities)



36

Examples of present Bayesian application at Lilly

• Multiple Bayesian designs (including adaptive) being used across the portfolio (Oncology, Endocrine, NSD)

• Bayesian inference used to determine Pr(technical success) at decision points for compound development.

• Used in Modeling and simulation• Global Product Safety: meta-analysis work

2006 FDA/Industry Workshop Advantages and Challenges of Bayesian Clinical Trials Increased Bayesian...

Documents

Transcript of 2006 FDA/Industry Workshop Advantages and Challenges of Bayesian Clinical Trials Increased Bayesian...