Adapting for Success: The Genesis of Adaptive Designs

Andy GrieveSVP Clinical Trials Methodology, Innovation Centre, Aptiv Solutions.

• 1

Outline

• Basic Principles of Adaptive Designs

• Why adaptive trials?

• Differences Between Early / Late Phase Adaptive

Designs

• Categories, benefits and regulatory feedback

• Practical Issues

2

Adaptive Ideas Are Not New

Biometrika, 1933

Ethical Design – concentrating on delivering the best treatment to the most patients Thall and Wathen (Eur J Cancer, 2007)

3

4

Basic Principles of Adaptive Designs

Terminology

Flexible

Multi-Stage

Response-Driven

Dynamic

Sequential

Self-Designing

Bayesian

Adap

tive

D

esig

ns

5

Adaptive Design: Definition

An Adaptive Trial uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. (PhRMA)

Validity4 providing correct statistical

inference:4 adjusted p-values, estimates,

confidence intervals4 providing convincing results to a

broader scientific community4 minimizing statistical bias

Integrity Pre-planning based on intended

adaptations

maintaining confidentiality of data

assuring consistency between different stages of the study

minimizing operational bias

6

General Structure – Dragalin (DIJ, 2006)

• An adaptive design requires the trial to be conducted in several stages with access to the accumulated data

• An adaptive design may have one or more rules:

– Allocation Rule: how subjects will be allocated to available arms– Sampling Rule: how many subjects will be sampled at next stage– Stopping Rule: when to stop the trial (for efficacy, harm, futility)– Decision Rule: the terminal decision rule and interim decisions pertaining to

design change not covered by the previous three rules

• At any stage the following stages can be redesigned taking account of all available data

7

Aspects of the Study Modifiable

• Number of Subjects• Study Duration• Endpoint Selection• Treatment Duration• Patient Population• Number of Treatments• Number of Interim Analyses• Hypotheses

8

The Role Of Adaptive Trials in the Drug Development Process

NewProduct

Standard Development Process

Phase 1 Phase 2 Phase 3

Adaptive Development Process

Option to:• Verify trial assumptions• Explore additional doses• Stop for futility early

Option to:• Select best dose• Submit application early• Stop for futility

9

Long periods of information “blackout”No opportunity to:

- verify trial assumptions- adjust dosing- make minor adjustments to trial design- stop for futility

Types of Adaptive Design: Learn

• Single ascending dose escalation designs • Up-and-Down and CRM to find MTD• Establish Proof-of-Mechanism or Proof-of-Target Modulation

First-in Human

• Two-stage adaptive approach in patients• 1st stage – to identify MTD• 2nd stage – to select dose and exposure levels (necessary cond.)

MAD and PoC

• Start with the highest feasible tolerated dose and placebo• If a pre-specified futility condition is satisfied => stop • Otherwise, open enrollment to lower doses

PoC and ADRS

• SAD or MAD combined with Biomarker-based Efficacy• To identify the Optimal Safe Dose

Seamless Phase I/II Design

• Finding a target dose (MED, EDp)• Response Adaptive Allocation• Covariate Adjusted Response Adaptive Allocation

Adaptive Dose Ranging Design

CRM: Continual Reassessment Method; MTD: Maximum Tolerated Dose; MAD: Multiple Ascending Dose; SAD: Single Ascending Dose; MED: Minimum Effective Dose; EDp: Dose achieving 100p% of maximum effect

10

Types of Adaptive Design: Confirm

• Sample size adjustment based on blinded or unblinded data:• Using nuisance parameter estimate• Using treatment effect estimate

Sample Size Reassessment

• Early stopping for efficacy, futility, harm or safety• Adjusting the number and/or timing of interim analyses• Increasing the maximum sample size

Adaptive Group Sequential Design

• Design combining the objectives of Phase II dose ranging study and confirmatory Phase III trial in a single protocol

• Dose selection at the interim analysis

Seamless Phase II/III Design

• Placebo run-in; Active control run-in; Dose titration• Adaptively enrich the population at the interim analysis• Enrich based on biomarker or clinical endpoint response

Population Enrichment Design

• Marker by Treatment Design• Targeted Design• Marker x TRT Design with Response adaptive allocation within strata

Drugs with Companion Diagnostics

11

Failure in Pharmaceutical Drug Development

Infectious D

iseases

Endocrine

AutoImmune

Respiratory

Neurology CV

Oncology

0%20%40%60%80%

Phase II Phase III

Therapeutic Area

Succ

ess R

ate

12

Arthriti

s & Pain CV

CNS

Infectious D

iseases

Oncology

Opthalmology

Metabolic Dise

ases

Urology

Women's Health

0%20%40%60%80%

Phase II Phase III

Therapeutic Area

Succ

ess R

ate

Kola and Landis (2004)Nature Reviews Drug Discovery

Biomed Tracker February 2011

Why are Adaptive Clinical Trials Essential for Development of New Products?

• Pharmaceutical industry facing a major pipeline challenge

– Fewer approvals, escalating development costs, tougher regulatory environment, re-imbursement hurdles, expiring patents for existing blockbusters

• Failure rate in Phase III estimated at 50%

• Traditional development paradigm is not sustainable

• Innovative designs are key priorities for improving R&D productivity and increasing the probability of success at Phase III.

13

15

The Need for Sample Size Re-Estimation in Late Phase Studies

Impact of key parameters on trial success

Delta: the treatment effect – measures the difference between drug group and placebo groupSigma: the standard deviation - measures the dispersion of patient responses around the meanPower (90%): probability of successError (5%): probability of success although no effect (aka type-1 error or false positive)Sample size needed, depends on all the above

16

SSSS SS

observed

design

Impact of Incorrect Assumptions: treatment effect and standard deviation

3.0

2.0

1.0

0.03.02.01.00.0

+ Primary Endpoint met - Primary Endpoint not met

Good news

observed

design

Data from 39 studies (phase 2 and 3) performed in a 2 year period

correct assumption

17

Uncertainties and Adaptive Insurance Solutions

• Uncertainty about treatment effect or variability of dataEarly stopping for futilitySample size re-estimation

• Uncertainty about dose arm to take forwardDose response adaptiveDose selection (seamless 2/3)

• Uncertainty about (sub)populationPopulation enrichment

• Drug simply doesn’t work

Early stopping for futility

18

Regulatory Environment

• EMA released (2007) the “Reflection paper” on adaptive designs in confirmatory trials– The emphasis is on control of Type I error rate– Consistency of treatment effect before and after adaptation

• PMDA started adaptive discussions and workshops with PhRMA since 2007– Wants to discuss pros and cons of adaptive designs– In summary (*), they will consider adaptive trials when there is a

compelling medical need unanswered otherwise.• Oncology (group sequential trials are more common)• Orphan drugs, rare disease indications• Severity of disease or difficulty of trial conduct• CNS with uncertainty about subjective endpoints and past trial results

(*) Ando et al. (2011). Adaptive Clinical Trials for new drug applications in Japan, European Neuropsycopharmacology,

19

Categories and Benefits

Adaptive Design adaptation Benefits FDA feedback

Phase 1 CRM*Dose escalation

Choice of Next Dose Better selection of MTD, less safety/efficacy issues

Phase 2 Adaptive dose finding

Change of randomization fraction

Better dose selection, less phase 3 failure, more $value at end of phase 2 Combine objectives of 2 trials in 1

Sample size re-estimation + Futility(Un-) Blinded

Increase sample sizeEarly stopping

Save underpowered trials, less phase 3 failure, verify trial assumptions and correct = buy an insurance policy, re-allocate resources faster if futile

Phase 2/3 combined

Drop dose Save time and patients, select the optimal dose

Population enrichment in phase 3

Change inclusion criteria, focus on subgroup

Win on the responder subgroup if drug not efficacious on the whole population

FDA

FDA

FDA

*CRM: Continual Re-assessment Method

FDAEMA

CbC

CbC

*CbC: Case by Case

CbC

CbC

20

21

Phase I Dose-Escalation Designs in Oncology

22

The Background (Oncology)

• Given several doses of a new compound, determine an acceptable dose for treating patients in future trials

• Assumptions– Definition of Dose Limiting Toxicity (DLT)– Definition of Maximum Tolerated Dose (MTD)

• Prob ( DLT | MTD) = *– Prob (Response) with dose A)– Prob (Toxicity) with dose B)

• These conflict : A) is good; B) is bad

Adaptive Dose-Escalation in Phase I Studies

Algorithmic Model-Based

Non-Bayesian

3+3 (Storer, 1989)• No dose-response assumptions

needed.• No explicit control of the target MTD-

rate.

Bayesian

mTPI (Ji et al, 2013)• Heuristic approach for determining a

dose with acceptable toxicity close to the MTD

• Overdose-control based on posterior probabilities of toxicity ranges

• Bayesian methodology for determining acceptance of dose levels

CRM (O’Quigley et al, 1990)• Dose selection based on the posterior

distribution of the dose-response.• No explicit over-dose controlBLRM (Neuenschwander et al (2008)• Dose selection is based on a two-

parameteric logistic model.• Overdose-control based on posterior

probabilities of toxicity ranges

23

Enter 3 patients

< 1/3 DLTs 1 /3 and < 2/3 DLTs > 2/3 DLTs

Add 3 patients

Escalate to Dose Level i + 1 Dose Level (i-1) is MTD

2/6 DLTs< 2/6 DLTs

Dose Level i

The 3+3 design has been used in approximately 95% of the published phase I oncology trials over the last two decades (Tourneau, 2009)

Schematic of a 3+3 Design

Candidate Dose Response Shapes

• tendency of the 3+3 designs to underestimate the target dose

• increases the chance of failure in terms of efficacy later

• Other characteristics - # number of patients with DLT’s, # patients dosed above the MTD …..

Probabilities of Selecting Appropriate Doses

Underdosing Target toxicity Overdosing0

20

40

60

80

100

MTD-selection - Scenario 1

BLOC mTPI CRM 3+3

Underdosing Target toxicity Overdosing0

20

40

60

80

MTD-selection - Scenario 2

BLOC mTPI CRM 3+3

Underdosing Target toxicity Overdosing0

20

40

60

80

100

MTD-selection - Scenario 3

BLOC mTPI CRM 3+3

27

Current MRC Developmental Pathway Funding Scheme (DPFS) Statisticians

“Good Cop” “Bad Cop”

It would be a good idea if the applicants considered a model-based approach

The applicants should use a model-based approach

28

Phase II / III Seamless Adaptive Design in Kidney Transplant Patients : Orphan Condition

Phase 2b Dose Selection Design Circa 1993

■ More Efficient● wider range of doses, smaller numbers of patients per group● followed by one large parallel group study focusing on the doses

showing promise in exploratory study.

2.5 mgs 10 mgsPlacebo 80mgs 120 mgs 160 mgsDose40 mgs

Redu

ction

from

Bas

elin

e25

20

15

10

5

0

29

0 0.5 1 1.5Dose

Clinician 1

Clinician 2

Clinician 3

Designing a Dose-Response Trial

31

Phase 2b Dose Response/Finding/Selection Designs2005-2010 / 2011-2013

2 3 4 5 >60

10

20

30

40

50

60

Number of Active Doses

Perc

enta

ge o

f Stu

dies

2005-2010

2011-2013

Seamless Phase II/III Designs

• Seamless design– A clinical trial design which combines into a single trial objectives which are

traditionally addressed in separate trials (operationally seamless)

• Adaptive Seamless design – A seamless trial in which the final analysis will use data from patients

enrolled before and after the adaptation(inferentially seamless)

• Primary objective – combine “dose selection” and “confirmation” into a single trial

• Key Benefits: Efficiency; faster and more informed decision-making• Key Challenges: Effective and Efficient Implementation

32

Efficiency of Adaptive Seamless Phase II/III Designs

33

Development Timeline

Inferentially Seamless Phase II/III trials

Dose B

Dose A

Placebo

Dose B

PlaceboPhase II

Operationally Seamless Phase II/III trial

Dose B

Dose A

Placebo

Dose B

PlaceboPhase II

Confirmatory AnalysisConfirmatory Analysis

Confirmatory Analysis

Interim analysis: Trigger for phase III

Dose C

Confirmatory Analysis

Dose C

Separate Phase II and phase III trials

Dose B

Dose A

Placebo

Dose B

PlaceboPhase II End of Phase III

Dose C

Confirmatory AnalysisI.

II.

III.

33

33

Phase II/III Seamless Design – checklist for feasibility

• The hypotheses are pre-defined and will not change• There is positive data from proof-of-concept studies• The remaining uncertainty primarily concerns dose• The primary endpoint for confirmation is pre-specified and will

be measured on all patients• Patient population will stay the same in both phases• The marketing formulation is available• There is sufficient animal data to allow longer drug exposure:

Phase II decision may be based on a biomarker believed to be predictive of the clinical endpoint for confirmation

34

Example: Seamless Phase II/III study in an Orphan Condition (on-going)

• Two-stage group sequential design with O’Brien & Fleming boundaries

• Dunnett intersection test (dose selection methodology)• Three doses of a drug with pre-specified effect sizes• Primary endpoint: Short-term response (0 or 1) (7 days)

Assumptions • One-sided type I error 0.025• Power 80%• Placebo rate 45% (to be reduced by treatment)

35

Comparison

• An inferentially seamless adaptive Phase II/III design with stopping after stage 1 only for efficacy using an O’Brien and Fleming boundary. The interim is assumed to be conducted at an information rate of 1/3rd

• An operationally seamless Phase II/III study with the same decision rule, and sample size (in stage 1), as the inferentially seamless design followed by a phase III study powered at 80%

36

0 200 400 600 800 1000 1200 14000

200

400

600

800

1000

1200

1400

ASN Inferentially Seamless Phase II/III Design

ASN

Ope

ratio

nally

Sea

mle

ss P

hase

II/II

I De

sign

Seamless Phase II/IIIIncreased Sample Size for Operationally Seamless Design

30% increase

37

Seamless Phase II/III study in an Orphan Condition

38

• Inferentially seamless design chosen• Design approved by EMA and FDA• Other case studies have shown similar savings• Orphan status would have been a potential

reason for PMDA to accept the design

39

Practical Issues in Execution of Adaptive Designs

Endpoints in Adaptive Trials

Indication Endpoint

Migraine Pain Control 2 hours

Kidney Transplantation Delayed Graft Function 7 days

Neuropathic Pain Pain Score 6 weeks

Acute Stroke Neurological Score 13 weeks

Oncology Months / Years

Delayed Response in Group Sequential TestsLisa Hampson & Chris Jennison (JRSSB, 2013)

• Develop designs which maximise the use of the pipeline data to increase the test’s power

Pipeline Data

Modelling Approach Basic Principle – Ian Marschner

enrolment times

parti

cipa

nt

start end of accrual interim analysis end of study

first measurement second measurement third measurement

6

5

4

3

2

1

Link to Missing Data

• Little & Rubin (2002) – “Analysis of Missing Data”– the concept of “monotone missing data”– for K measurements Y1,Y2,…,YK

– Yj+1,Yj+2,…,YK are missing when Yj is missing

• The advantage of this structure is that the likelihood can be partitioned allowing simplified estimation

“Simple” Illustration of Efficiency Gains

• Assumptions– Normally distributed data, – Two treatments– n patients per treatment have data on short-term and long-term

endpoints– m patients per treatment have data on short-term endpoint only– Variances equal and known, correlation known

“Simple” Illustration of Efficiency Gains

• Assumptions– Normally distributed data, – Two treatments– n patients per treatment have data on short-term and long-term

endpoints– m patients per treatment have data on short-term endpoint only– Variances equal and known, correlation known

Nyy

~.12

.11

m

Nx2

11.1 ,~

m

Nx2

21.2 ,~

1

1,~

2

12

11

.12

.11

nN

yy

1

1,~

2

22

21

.22

.21

nN

yy

Short-term

Long-term

Short-term

Long-term

Treatment 1

Complete data :

Short-term only:

Treatment 2Complete data :

Short-term only:

“Simple” Illustration of Efficiency Gains

• Interested in estimating long-term treatment effect : 12 - 22

nV

yy2

22121

.22.122212

2)ˆˆ(

ˆˆ

Using only long-term data

)(22)ˆˆ(

ˆˆ

222

22122

.21.1121.22.122212

mnnm

nV

yyxxmn

myy

Using all data (long-term and short-term)

cf. Galbraith & MarschnerSection 2.3 generalized to k measurements

Efficiency = 100V1/ V2 ; m=an

a

100

150

200

250

300

350

400

0 0.2 0.4 0.6 0.8 1

Effici

ency

(%)

108

6

4

2

1

½¼

Data from Copenhagen Stroke Database (Jorgensen et al, 1995):Mean Response by Length of stay in the Acute Stroke Unit (0-12 Weeks)

12 1212

12 12 12 12 12 12 12 12 12 12

11

1111 11 11 11 11 11 11 11 11 11

1010

1010 10 10 10 10 10 10 10

99

99 9 9

9 9 9 9

8 88

88 8 8 8

01

1

2

22

3

3 3 3

7

7 77 7 7 7 7

6

66

66 6 6

4

44 4 4

5

55 5

5 5

8

25

30

35

40

45

50

55

60

0 2 4 6 8 10 12 14

Week (Post Stroke)

Mea

n S

cand

inav

ian

Stro

ke S

core