Finding Studies Adaptive Designs in Dose- · 2007. 11. 14. · Webinar 2: Adaptive designs in...

Fall 2007 web-based training series

Introduction (Alex Dmitrienko, Lilly)

Web-based training program

• http://www.amstat.org/sections/sbiop/webinarseries.html

Webinar 2: Adaptive designs in dose-finding studies

• Brenda Gaydos (Lilly), José Pinheiro (Novartis), Chris Coffey (University of Alabama-Birmingham)

Presentation slides

• http://www.biopharmnet.com/doc/doc03002.html

Discussion thread

• http://biopharmnet.com/forum/viewtopic.php?t=102

Sponsors: Biopharmaceutical Section of ASA ,

Adaptive Designs in Dose-

Finding Studies

Four-Part Web-based Training Series

Webinar 2

Adaptive Design Working Group

Adaptive Dose Ranging Studies Working Group


PRESENTERS

Christopher S. Coffey, PhD

University of Alabama at BirminghamEmail: [email protected]

Brenda Gaydos, PhD

Eli Lilly and CompanyEmail: [email protected]

José Pinheiro, PhD

Novartis PharmaceuticalsEmail: [email protected]


• Co-Chairs:

Michael Krams

Brenda Gaydos

• Member Authors:

Keaven Anderson

Suman Bhattacharya

Alun Bedding

Don Berry

Frank Bretz

Christy Chuang-Stein

Sylva Collins

Vlad Dragalin

Paul Gallo

Brenda Gaydos

Michael Krams

Qing Liu

Jeff Maca

Inna Perevozskaya

Jose Pinheiro

Judith Quinlan

• Members:

Zoran Antonijevic

Roy Baranello

Michael Branson

Carl-Fredrik Burman

Nancy Burnham

Daniel Burns

Bob Clay

Chris Coffey

David DeBrota

Alex Dmitrienko

Jennifer Dudinak

Greg Enas

Richard Entsuah

Parvin Fordipour

Sam Givens

Ekkehard Glimm

Andy Grieve

Shu Han

• Members (cont.):

David Henry

Melissa Himstedt

Tony Ho

Roger Lewis

Gary Littman

Cyrus Mehta

Wili Maurer

Allan Pallay

Michael Poole

Bob Parker

Yili Pritchett

Jerry Schindler

Jonathan Smith

Don Stanski

Joel Waksman

Bill Wang

Gernot Wassmer

ADAPTIVE DESIGNS WORKING GROUP


• Co-Chairs:

José PinheiroRick Sax

• Member Authors:

Björn Bornkamp

Frank Bretz

Alex Dmitrienko

Greg Enas

Brenda Gaydos

Chyi-Hung Hsu

Franz König

Michael Krams

Qing Liu

Beat Neuenschwander

Tom Parke

Amit Roy

Frank Shen

• Members:

Zoran Antonijevic

Vlad Dragalin

Parvin Fordipour

Marc Gastonguay

Bill Gillespie

Frank Miller

Inna Perevozskaya

Ashish Sanil

Jonathan Smith

ADAPTIVE DOSE-RANGING STUDIES WG


OUTLINE

I. Introduction and Methods for Early Exploratory Studies (~35 min.)

II. Methods for Late-Stage Exploratory Development(~50 min.)

III. Simulations and Conclusions (~35 min.)


I. Introduction and Methods for Early

Exploratory Studies

Outline:

• Overview of adaptive designs

• Summary of major philosophies regarding definition of maximum tolerated dose (MTD)

• Conventional 3+3 designs

• Model-based designs


OVERVIEW

Infinite number of adaptive design possibilities:

• Many aspects of the study can be changed:

- sample size - final test statistic

- primary endpoint - inclusion/exclusion criteria

- number of treatment arms - randomization procedure

- Number of interim looks - goal: superiority to non-inferiority

• Define objective of the adaptation and the design elements to adapt.


OVERVIEW

For this course, we focus on adaptive dose-response methods.

Such adaptive designs:

• Offer more efficient ways to learn about dose response

• Provide more information on dose-response profile earlier in development.

• Guide decision making on whether to continue program and, if so, which dose to select for further development

• Aim to increase probability of technical success by taking correct choice of dose forward for further study.


MTD - DEFINITION

Phase I clinical trials typically want to determine some maximum tolerated dose (MTD).

Accurate determination of the MTD is very important since the dose established as the MTD will be used for further testing in later phases.

Passing on too low of a dose may jeopardize a potentially useful drug

Passing on too high of a dose puts patients in later phase trials at risk


MTD - DEFINITION

Two major philosophies regarding MTD definition:

Dose that, if exceeded, would put patients at ‘unacceptable risk’ of toxicity.

• Treat the MTD as being observed from the data

• Vague from statisticians point of view since ‘unacceptable risk’may not be defined quantitatively

Specifying ‘unacceptable risk’ as a probability.

• Treat the MTD as an unknown parameter of a monotonic dose response curve.

• The MTD is estimated corresponding to a specified probability.


MTD - DEFINITION

1) Conventional up-and-down designs

• Such as 3+3 designs for cancer

2) Model-based designs where MTD is a quantile to be estimated

• Random walk rule

• Bayesian methods

These two definitions lead to two different approaches for designing phase I clinical trials:


CONVENTIONAL 3+3 DESIGNS

Conventional 3+3 methods employ an ad-hoc approach to screen dose levels and identify the MTD.

Toxicity is defined as a binary event and patients are treated in groups of three, starting with the initial dose.

Algorithm iterates moving dose up or down depending on the number of toxicities observed.

No estimation in a traditional sense is involved.

The MTD is a statistic identified from the data - highest dose studied with less than, say 1/3 toxicities (i.e., 0 or 1 dose-limiting toxicities out of six patients).


0 2 or moreCount

Events

Treat 3 patients at dose

Start at the lowest

reasonable dose

Increase dose to next level

Treat 3 additional patients

at dose

Count

Events

Decrease dose or stop and

select lower dose

1

0 1 or more




0%

20%

40%

60%

80%

100%

0.0 0.2 0.4 0.6 0.8 1.0

True "p"

Ch

an

ce o

f "S

tep

pin

g U

p"

Even with a

30% chance of

an “event”

there is still a

50% chance of

stepping up!



Strengths:

Simple to implement and understand

Requires no computer program

Familiar to many clinicians

Drawbacks:

Tend to treat many patients at low, ineffective doses

Often provide poor estimates of MTD (i.e., large uncertainty) -probability of stopping at incorrect dose higher than perceived

Hence, unsafe or non-efficacious doses may be advanced to Phase III trials.


CRM

Continual Reassessment Method (CRM):

Originated as a Bayesian method for phase I cancer trials of cytotoxic agents.

For a pre-defined set of doses and a binary response, estimates MTD as the dose level that yields a particular target proportion of responses (e.g., TD20).

Assumes a particular model (such as logistic function)

Assignment of doses converges to the MTD.

See Garrett-Moyer (2006) for an excellent tutorial.


CRM

The method assumes that the probabilities of both efficacy and toxicity increase with increasing dose.

The method also assumes that toxicity can be defined as a binary outcome.

The “acceptable” toxicity rate is explicitly defined and the MTD is the highest (most efficacious) dose with acceptable toxicity.

Similar designs can be used to explore dose-efficacy relationships (for agents that are non-cytotoxic).


CRM

The method begins with an assumed a priori dose-toxicity curve and a chosen target toxicity rate.

The first patients are assigned the dose most likely to be associated with the target toxicity level.

The estimated dose-toxicity curve is refit (i.e., the posterior distribution of the model is updated) after each patient’s outcome has been observed.

Hence, the updated curve is shifted slightly up or down depending on whether the patient experienced a dose-limiting toxicity.


CRM

The next patient is assigned the dose closest to the MTD based on the updated dose-toxicity curve (posterior distribution).

Patients continue to be treated until some pre-defined level of certainty is achieved or pre-defined stopping criteria are met.

Once the stopping criteria is achieved, the final dose is selected as the MTD.


CRM

For example, consider the following curve:

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20

Dose

Even

t R

ate

If target level of toxicity is 10%, then dose level 5 would be the optimal starting dose.


CRM

An example of how the CRM might work:

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20

Dose

Even

t R

ate


CRM

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20

Dose

Even

t R

ate

An example of how the CRM might work:

Final Dose


CRM

The implementation of a CRM requires a substantial collaboration between the investigator and statistician.

This collaboration is important in order to determine:

The dose-toxicity model to use

• one parameter logistic, two parameter logistic, etc.

The target rate of toxicity (or response)

Stopping rules

• Fixed # of patients

• Fixed # of patients treated at a dose


CRM

When publishing results from a CRM trial, typical to display:

The recommended dose (MTD) for a future trial, along with some estimate of the variability surrounding the MTD estimate.

A table that shows how the CRM progressed, including:

• Number of dose-limiting toxicities for each cohort

• Estimated dose at end of each cohort


CRM

Strengths:

“Learns” from information gained at early time points in the study – all patients studied contribute to the estimated dose.

Less likely to treat patients at toxic doses – tends to incur fewer dose-limiting toxicities.

More likely to treat patients at efficacious doses

Can more accurately estimate the MTD as compared to standard 3+3 designs


CRM

Drawbacks:

Mathematical and statistical complexities make it difficult for many clinical investigators to understand.

Properties must be assessed via simulation.

Early on, large dose escalations can occur based on little information which may cause more patients to be treated at unsafe doses.

Dosing first patients at level deemed appropriate by a priori curve may be worrisome due to uncertainty surrounding this curve.


MODIFIED CRM’s

To address some of the concerns with the original CRM, Several modified CRM approaches have been developed and implemented:

Always start at the lowest dose level under consideration

Enroll 2-3 patients in each cohort

Proceed as a standard 3+3 dose escalation design in the absence of dose-limiting toxicities.

Any given dose escalation cannot increase by more than one level.


MODIFIED CRM’s

Strengths:

Mathematical model is not solely responsible for determining dosage increases – restricted by design.

Starting dose can be chosen as with a traditional design – start dosing at the lowest level

Drawbacks:

Mathematical and statistical complexities make it difficult for many clinical investigators to understand.

Properties must be assessed via simulation.


OTHER BAYESIAN DESIGNS

1) Escalation with overdose control

• Similar to CRM, but addresses ethical need to control probability of overdosing

2) Designs based on Bayesian decision theory

• Next dose assignment is determined by maximizing the gain function, which is sequentially updated after each response

3) Bayesian D-Optimal designs

• Introduces formal optimality criterion (D-optimality) minimizing the determinant of the variance-covariance matrix of the model parameter estimates


SUMMARY

Standard 3+3 designs were not designed with the intention of producing accurate estimates of a target quantile.

Bayesian model-based methods (such as the CRM) provide better estimates of the MTD and dose-response curve.

However, such methods are complicated to explain to non-statisticians and computationally challenging to implement.

The key to their usefulness lies in the packaging of these methods in user-friendly software that runs quickly and is well-documented.


1. Multiple Comparison (MC) Approaches

Frequentist based

Some approaches imbed Bayesian methods within study stages

2. Model Based Approaches

Bayesian

Frequentist (D-optimal Criterion)

II. Adaptive Dose-Response Methods for

Late-Stage Exploratory Development


PART 1: MC APPROACHES

Objectives

Identify a dose-response relationship (trend test)

Select dose(s) effective relative to a control (pairwise comparisons)

Hypothesis tests based on contrasts between doses

Dose is a categorical variable

Advantages

Easy to implement

Limited assumptions needed

Useful if too few doses to enable modeling


DISADVANTAGES

Designed for estimation and testing of doses used in the trial

Hypothesis testing typically requires high sample sizes per dose group

Feasibility limits number of doses explored

May identify if dose response exists BUT

Does not provide information on the dose response profile

Does not provide for quantitative estimates and precision of targeted dose(s) of interest such as MED or ED95


Approaches

Extending the classical group sequential framework

Combination function approaches (foundation in meta-analysis)

Types of adaptations:

Add or drop doses

Sample size reassessment for future stages

Early stopping for futility or efficacy

Seamless shift across development phases

MC FRAMEWORK FOR ADAPTIVE METHODS


Stallard & Todd (2003)

Extended classical group sequential designs to multiple treatment arms

Identify best treatment based on a maximum standardized test statistic

GROUP SEQUENTIAL DESIGNS


Approach

Trial analyzed in a series of independent stages

Very flexible

– Do NOT have to define what you will adapt in advance

– Bayesian decision theoretical approach can be used

– DO have to define a-priori how you will combine the test statistics from the stages to make inference

Controls family-wise type I error rate

Adjustments needed for inference (Posch et al. 2005)

Multiplicity adjusted p-values for dose-control comparisons

Point estimates and CI adjusted for early stopping and treatmentselection

ADAPTIVE TREATMENT SELECTION BASED ON

COMBINATION TESTS


Stage wise tests

Independent observations between stages

Let p1, p2 be p-values from stages 1 and 2 respectively

A-priori define at minimum

Combination function

Stage wise and overall alpha levels

More than 18 different combination functions (Becker, 1994)

Commonly used functions

Fisher’s: C(p1,p2) = p1*p2

Inverse Normal: C(p1,p2) = -w1N-1(1-p1) - w2N

-1(1-p2)

THEORETICAL BACKGROUND


COMBINING P-VALUES: FISHER’S METHOD

Under the null: Pi ~ U(0,1) i.i.d.

Let ; then i.i.d.

and

To test the Global Null Hypothesis: H0: H01 H02

Note: Pr(H01 H02) = P1·P2

Hence; compare -2(ln p1 + ln p2) to critical value

2lni iX P2

2~i dfX

2

2

1

~n

i n df

i

X

2

4 (1 )df


Note P may only be approximately uniform [0,1] under the Null:

IF individual hypotheses are composite, or if responses are discrete

Jennison & Turnbull (2005); Robins, et al. (2000)

Two-stage procedure Bauer & Kieser (1999):

Weaker condition:

Distribution of P1 and conditional distribution of P2|P1

stochastically larger than or equal to the uniform distribution on [0,1]

ON DISTRIBUTION OF P-VALUE


Jennison & Turnbull (2005)

Inverse Normal: C(p1,p2) = -w1N-1(1-p1) - w2 N-1(1-p2)

Historically

Mosteller & Bush (1954): generalization based on fixed weights

– If

– Then

Interpretation concern– Weighting patient information unequally based on stage

INVERSE NORMAL FUNCTION

1

1

1( ... ) (1 ) ~ (0,1)k k kZ Z where Z N p N

k

2

1

1K

k

k

w

1

~ (0,1)K

k k

k

w Z N


Let

Then the z-statistic for pooled data equals:

Combination test statistic equals z-statistic for pooled data

– Invariant to partitioning of the data

– A function of the sufficient statistic (efficient)

– Sample size of stages must be fixed

Note: In general, the number of stages & weights can be adapted for K > 2

Fisher (1998) “variance spending”

– Spend of variance of Z statistic: study ends when sum is 1

INVERSE NORMAL FUNCTION (cont.)

1

K

k k

k

Z w Z

kk

nw

N

2

kw


Bauer & Kohne (1994)

The pre-specified combination test needs to be followed

Properties only hold if followed

e.g. Cannot decide to treat Stage 1 as internal pilot (even if no adaptation is made for Stage 2)

“Protocol has to describe which types of adaptation are

intended.”

Conclusions depend on types of adaptations

Ad-hoc adaptations (even if family-wise Type I error preserved) can make interpretation difficult

Estimates may be biased or intractable

POTENTIAL ABUSES


CombinesClosed testing procedure

A multiplicity adjustment procedure (e.g. Bonferroni-Holm min P, Simms)

A combination test procedure (Fishers, Inverse Normal)

Same approach can be used for seamless designsStages can span Phase II/III

APPLICATION TO DOSE FINDING


Assess design options/power via simulations

Power is a function of unknown dose response

In two stage approach with fixed sample sizes, inverse normal combination function is efficient

Model based approaches may be more efficient (but also more complex)

Resulting estimates can be biased

Recommend assessing via simulation

Last resort, use the last stage for estimation purposes

DO NOT ABUSE

Follow required pre-specified rules

Describe possible adaptations in protocol

RECOMMENDATIONS


Objectives:

Estimate dose-response

Identify optimal (target) dose(s)

ASSUMES a functional relationship between the dose and response (Parametric & Non-parametric model-based approaches)

Advantages

Quantitative estimates, such as ED95, can be made from the model

Provides quantitative information on the dose response profile useful in planning future studies

PART 2: MODEL BASED APPROACHES


More sensitive to model assumptions (parametric model)

Estimation and analysis more complex (e.g. non-linear models)

Sample size calculation usually requires simulations

Typically requires more doses to implement

DISADVANTAGES


Focus

Efficient learning: More observations at doses that better inform the dose-response and or the target dose(s)

Effective treatment of patients in the trial: maximize patient exposure to effective doses/Tx

Approaches

Bayesian (predictive probabilities)

Frequentist (optimal experimental design)

ADAPTIVE MODEL BASED APPROACHES


Dose-response model: parametric or non-parametric

Longitudinal models: predict final patient outcome data at interims

Prior distributions on model parameters

Decision making components may include:

Dose allocation rules

Stopping rules

Decision rules can be highly flexible:

Minimize expected loss function

Maximize expected utility function ( - loss function)

BAYESIAN APPROACH: COMPONENTS


West and Harrison (1997): Bayesian Forecasting and Dynamic Models

A piece-wise linear model

Smoothed transitions in the dose-response slope across the doses

Does not restrict the shape of the dose response curve

Developed for analysis and forecasting of time series data

Other non-parametric models

Splines, Kernel Methods

NORMAL DYNAMIC LINEAR MODEL


Assumptions

Response at each dose normally distributed about a mean

Change in mean between adjacent doses can be predicted by a simple linear model

Variability decomposed into two components

Observational variability for the patient response about the mean for the given dose

System variability around the linear model that relates the

adjacent means

NDLM (cont.)


NDLM (cont.)

Priors placed on:

µk (mean at dose k) H (smoothing parameter)

(slope parameters) 2 (observational variance)

Let Rik be the ith patient response at dose k, and Dk

represent the kth dose with mean µk.

Observation Equation:

System Equations:

2| ~ (0, )ik k k ik ikR D where N

2

1 1 ~ (0, )k k k k kwhere N H

2

1 ~ (0, )k k k kwhere N H


4-PARAMETER LOGISTIC MODEL

4

1 22

3

1 ( )i i

i

RD

Patient indicator

Patient response

Level of drug

Response at 0 drug

Max. attributable effect of drug + 1

Dose producing response half way between 1 and 2

Related to steepness of slope

Random error for patient I {often iid N(0,1) }

1

2

3

4

i

i

i

i

R

D

1-1 Comparison to Emax Model

(Di)-1= Di

4 = Hill Coef- 4 = Hill Coef

( 3)-1 = ED503 = ED50

1 - 2 = Emax1 - 2 = Emax

2 = E02 = E0

4> 0

4< 0

50

max0

ii i

i

D ER E

D ED


Requires monotonicity: increasing or decreasing

Minimum of 5 doses desirable (4-parameter model)

If highest dose < ED95

Estimates of Emax, ED50, and Hill Coefficient (gamma) impacted: high coefficient of variation & bias

Fit in data range usually good

Bayesian approach

Strong priors might be assumed for Emax if highest dose thought to be less than ED95

4-PARAMETER LOGISTIC MODEL (cont.)


Targets estimation of the overall dose-response

Formal optimality criterion (D-optimal)

Minimize determinant of the variance covariance matrix of the model parameter estimates (maximizes information)

Allocates patients (sequentially or group sequentially) to provide the most information

Typically keep allocation of placebo constant

Wide class of dose-response models are applicable

e.g. Four-parameter logistic model

D-OPTIMAL: AN OPTIMAL DESIGN APPROACH


One approach (ADRS WG white paper)

Allocate equally across doses for first cohort

Fit model

Based on this model, determine optimal allocation ratio for nextcohort of patients to maximize information

Bayesian D-Optimal

Place a prior distribution on the model parameters

After each cohort, calculate the posterior distribution

Similarly, update allocation ratio to minimize determinate of the variance co-variance matrix of model parameters

D-OPTIMAL (cont.)


Neuropathic Pain

Minimum Clinical Significance:

Average Daily Pain Score (ADPS)

Ranges (0 no pain, 10 worst pain)

1.5 difference from placebo change from baseline

Design PoC study to select future dose(s) Phase III

12 fold dose range

Dose-response unknown…may be inverted-U shaped

Positive control desirable for assay sensitivity

Too costly to explore dose-range?

CONSIDER


Resp

on

se

Dose

NotInformative

Informative

FIXED DOSE DESIGN


• Pfizer: Smith, Jones, Morris, Grieve, Tan (2006)

1 wk

Lead-In

4 wks

Double Blind Treatment

1 wk

Follow-up

7 Doses

Positive Control

Placebo

Max n=35 per arm

Type I error < 5%

Power ~ 80%

ADAPTIVE PoC CASE STUDY


Objective: select doses for Phase 3

Dose-Response Model: NDLM

Decision Making Components:Dose Allocation Rule: equal allocation

Stopping Rules (2 interims)

– Continue the study if at least one dose X has:

Pr (Effect at dose X > 1.5) > 0.80

– Drop up to 2 non-efficacious doses at each interim where:

Pr ( Effect at dose X < 1.5 ) > 0.80

ADAPTIVE FEATURES


Trial stopped at first interim (flat dose-response)

Approximately $2M saved due to stopping early

Rough comparison to fixed design with pairwise comparison of each dose to placebo

Approximately 3-4 times larger

No early stopping

Controlling for multiple comparisons

Type I error, 1-sided, 10%

RESULTS


Objectives:

Identify target dose (ED95)

Estimate dose response

Dose-Response Model: NDLM

Decision Making Components:

Dose Allocation Rule (maximized utility function)

Stopping Rules (decision analytic)

EXAMPLE: ASTIN (Krams et al. 2003)


Maximize Utility Function: Allocate patients to maximize information about ED95

Minus the variance of the predicted mean response at the ED95

Includes uncertainty in ED 95 dose & in the dose response

Function of future patient data

Determining next patient assignment

Calculate the expected utility for each possible dose assignment

– Expectation over the posterior predictive distribution for the data yet to be observed

– Ongoing patient data predicted from earlier data using a longitudinal model (that gets updated during the study)

Assume next patient is last patient

Assign dose that is expected to result in the smallest variance

Maintain blind by selecting randomly within 5% of optimal dose

DOSE ALLOCATION RULE


Trial stopped for futility

Savings

Dollars: $3.5M direct grant costs

Time: termination decision at least 6 months earlier

RESULTS


Phase II dose ranging study

Schizophrenia

Objective

Confirm positive POC study

– 3 arm: High dose, Active (assay sensitivity), Placebo

Explore lower doses

Determine dose(s) Phase III

Dose range 8 fold

4 doses

Primary Measure

PANSS total score at 6 weeks

EX: ADAPTIVE DESIGN NOT RECOMMENDED


Subjective primary measure / 20 sites

Significant effect due to site

Desirable to stratify by site

Long term outcome relative to expected enrollment rate (no biomarker)

Narrow dose range well covered by 4 doses

High dose effective, but may not be near Emax

STUDY CHARACTERISTICS


Fixed design with equal allocation

Adaptive allocation

Bayesian D-Optimal Criterion (4 parameter logistic model)

Allocation adapts to increase efficacy of estimates of model parameters

Stopping Rules (4 interim analyses)

Stop for Futility: If predicted mean difference high dose vs placebo > -5, with 95% confidence

Stop for Efficacy: If predicted mean difference low dose vsplacebo < 0, with 95% confidence

DESIGNS COMPARED


No compelling advantage to adaptive randomization over fixed allocation

Adaptive randomization favored slightly more subjects randomized to placebo and approximately equal allocation to other doses

Fixed design would be slightly more powerful for pairwise comparisons with unequal allocation & not effect dose-response estimation adversely (2:1:1:1:1)

Efficiencies over-estimated since perfect information was assumed in simulations for adaptive allocation

Use of parametric dose-response model (unknown dose-response)

Additional resources/complexity not warranted

RECOMMENDATION


Consider routinely assessing appropriateness of adaptive designsin exploratory development

Asses potential gains against those of standard fixed designs

Balance complexity with potential gains

Trial simulations typically needed

Fine tune design

Assess operating characteristics

Recommended even when ONLY considering a fixed trial design

Consider Seamless PoC/Phase 2 dose-response studies

Recommend model based approaches

More informative of dose response profile than MC

Critical to assess model assumptions

Non-parametric models less restrictive

CONCLUDING REMARKS


III. Simulations Comparing Adaptive and

Non-Adaptive DF Methods

Outline:

• Evaluating statistical operational characteristics of complex DF designs and methods

• Comparing DF designs and methods: PhRMA’s Adaptive Dose Ranging Studies working group simulation study

• Conclusions from ADRS WG simulations


MOTIVATION

Evaluation of operational characteristics (OCs) of proposed statistical methods is a critical step in designing a clinical trial – comparison of methods

The OCs include the power to detect signals of interest, the precision of estimates for quantities of interest, expected duration, etc in particular, used to determine sample size and number of arms

Complexity of adaptive dose finding designs and other non-traditional dose finding methods typically no closed form expressions for OCs metrics

Simulation-based evaluation needs to be employed


KEY GOALS OF DF TRIALS

Typical goals of Phase II trials:

Determine evidence of dose response (DR) signal, i.e., if average response changes with dose level – proof-of-concept (PoC)

Select target dose(s) for confirmatory phase – typically MED; other targets also used (e.g., maximum useful dose)

Estimate DR profile – usually for efficacy, but safety of increasing interest

These goals determine the design of the study and the operational characteristics that need to be evaluated


SIMULATING DF TRIALS

Trial simulation is the main tool for evaluating study OCs; itneeds to properly incorporate multiple factors in study:

Type: parallel groups, cross-over, titration, etc

Available doses, inclusion of active control(s)

Dose allocation scheme (fixed vs. adaptive)

If adaptive, frequency and timing of adaptations (and algorithm for recalculating allocation ratios)

Dose response profile(s):

more than one should be used to assess sensitivity

flat dose response should be included to assess Type I error andimpact on dose selection


SIMULATING DF TRIALS (CONT.)

Response variables: type (e.g., continuous, binary, count, ordinal); distribution (e.g., normal, Poisson)

Variance and covariance parameters (e.g., within- and between patient variances, between-site variances)

Sample size (e.g., expected, maximum)

Sensitivity analysis: impact of changes in assumed parameters/models/design on OCs (highly recommended)

Drop-out and missing data models (e.g., time to drop-out)

Patient accrual process (e.g., rates, uniformity over time)

Stopping rules, if any (e.g., futility, efficacy)


PhRMA ADRS WG

Adaptive Dose Ranging Studies (ADRS) WG: formed, with others, asresult of survey to identify key drivers of poor performance in pharmaindustry poor understanding of DR indicated as a leading cause for high attrition in late development.

Investigate and develop designs and methods for efficiently learningabout efficacy and safety DR profiles

Evaluate statistical OCs of alternative designs and methods (adaptive and fixed) to make recommendations on their use

Increase awareness about ADRS, promoting their use, when advantageous

Comprehensive simulation study comparing ADRS to other DF methods, quantifying potential gains


SUMMARY OF DESIGN AND ASSUMPTIONS

Proof-of-concept + dose-finding trial, motivated by neuropathic pain indication (conclusions and recommendations can be generalized)

Key questions: Whether there is evidence of dose response and, if so, which dose level to bring to confirmatory phase and how welldose response (DR) curve is estimated.

Primary endpoint: Change from baseline in VAS at Week 6 (continuous, normally distributed)

Dose design scenarios (parallel arms):

- 5 equally spaced dose levels: 0, 2, 4, 6, 8

- 7 unequally spaced dose levels: 0, 2, 3, 4, 5, 6, 8

- 9 equally spaced dose levels: 0, 1, …, 8

Significance level: one sided FWER = 0.05

Sample sizes: 150 and 250 patients (total)


DOSE RESPONSE PROFILES


DF METHODS USED IN SIMULATONS

Traditional ANOVA based on pairwise comparisons and multiplicity adjustment (Dunnett)

MCP-Mod combination of multiple comparison procedure (MCP) and modeling (Bretz, Pinheiro, and Branson, 2005)

MTT: novel method based on Multiple Trend Tests

Bayesian Model Averaging: BMA

Nonparametric local regression fitting: LOCFIT

GADA: Dynamic dose allocation based on Bayesian normal dynamic linear model (Krams, Lee, and Berry, 2005)

D-opt: adaptive dose allocation based on D-optimality criterion


SELECTED SIMULATION

RESULTS

More detailed results given in the ADRS WG’s White Paper, available at http://biopharmnet.com/doc/doc12005.html


POWER TO IDENTIFY DR


DOSE SELECTION UNDER FLAT DR


DOSE SELECTION UNDER ACTIVE DR


CORRECT TARGET DOSE INTERVAL


DOSES SELECTED – LOGISTIC, N=150


SAMPLE PRED. DR – LOGISTIC, N=150


ADRS WG CONCLUSIONS

Detecting DR is considerably easier than estimating it

Current sample sizes for DF studies, based on power to detect DR, are inappropriate for dose selection and DR estimation

None of methods had good performance in estimating dose in the correct target interval: maximum observed percentage of correct interval selection – 60% larger N

needed

Adaptive dose-ranging methods (i.e., ADRS) lead to gains in power to detect DR, precision to select target dose, and to estimate DR – greatest potential in the latter two


ADRS WG CONCLUSIONS

Model-based methods have superior performance compared to methods based on hypothesis testing

Number of doses larger than 5 does not seem to produce significant gains (provided overall N is fixed) trade-offbetween more detail about DR and less precision at each dose

In practice, need to balance gains associated with adaptive dose ranging designs against greater methodological and operational complexity


IV. Overall Conclusions and

Recommendations


CONCLUSIONS & RECOMMENDATIONS

Adaptive, model-based dose finding designs should be routinely considered for use in drug development (Early Development, PoC, Dose Ranging) can lead to substantial gains in efficiency over traditional methods

Dose assignment algorithm should be prospectively andclearly specified in study protocol

Trial simulations should be used to fully evaluate operational characteristics of design prior to study start

Seamless approaches should be considered to improve efficiency, especially between PoC (Ph. I/IIa) and dose ranging (Ph. IIb)


CONCL. & RECOMMENDATIONS (CONT.)

Sample size calculations for adaptive DF designs should take into account the precision of target dose estimates and, more broadly, the accuracy of the decision(s) to be made from the study

Early stopping rules, for efficacy and safety, should be implemented, when feasible, to allow greater efficiency gains in adaptive design

Potential gains associated with adaptive approaches should always be contrasted to additional complexity and costs related to their implementation – not a panacea


CONCL. & RECOMMENDATIONS (CONT.)

Greater usage of these adaptive DF designs should be encouraged and will require:

• Good quality software packages with well documented code and examples for implementing approaches and conducting simulations needed to evaluate operating characteristics of these methods.

• A greater understanding of the strengths and weaknesses of these approaches (hopefully, this course has helped out along this regard)

• More published examples of studies that have utilized these methods.


Bauer P and Kieser M (1999). Combining different phases in the development of medical treatments within a single trial. Statistics in Medicine, 18:1833-1848.

Bauer P and Kohne K (1994). Evaluation of experiments with adaptive interim analysis. Biometrics, 50:1029-1041.

Fisher LD (1998). Self-designing clinical trials. Statistics in Medicine, 17:1551–1562

Gaydos B, Krams M, Perevozskaya I, Bretz F, Liu Q, Gallo P, Berry D, Chuang-Stein C, Pinheiro J, Bedding A (2006). Adaptive Dose-Response Studies. Drug Information Journal, 40(6): 451-461.

Garrett-Moyer E (2006). The continual reassessment method for dose-finding studies: A Tutorial. Clinical Trials, 3: 57-71.

Jennison C and Turnbull BW (2005). Meta-analyses and adaptive group sequential designs in the clinical development process. Journal of Biopharmaceutical Statistics, 15: 537-558.

SELECT REFERENCES


Krams M, Lees KR, Hacke W, Grieve AP, Orgogozo JM, Ford GA for the ASTIN Study investigators. (2003). ASTIN: An adaptive dose-response study of UK-279,276 in acute ischemic stroke. Stroke, 34: 2543-254.

Posch M, Koenig F, Branson M, Brannath W, Dunger-Baldauf C, and Bauer P (2005). Testing and estimation in flexible group sequential designs with adaptive treatment selection. Statistics in Medicine, 24: 3697-3714.

Robins JM, van der Vaart A, and Ventura V (2000). Asymptotic Distribution of P Values in Composite Null Models. Journal of the American Statistical Association, 95: 1143-1156.

Smith MK, Jones I, Morris MF, Grieve AP, and Tan K (2006). Implementation of a Bayesian adaptive design in proof of concept study. Pharmaceutical Statistics,5:39-50.

Stallard N and Todd S (2003). Sequential designs for phase III clinical trials incorporating treatment selection. Statistics in Medicine, 22: 689-703.

West M, Harrison PJ. (1997). Bayesian Forecasting and Dynamic Models. Springer-Verlag, New York.

SELECT REFERENCES (CONTINUED)


BACK-UP

NOTE: The following slides will not be covered in the presentation, but are added to provide

additional details.


Following example taken from PhRMA Adaptive Design Working Group training presentation

Titled: Adaptive Seamless Designs for Phase IIb/III

Clinical Trials

Author: Jeff Maca, Ph.D., Novartis

Full set of this and other training slides can be found at the following open access WEB site:

http://biopharmnet.com/doc/doc12004.html

ADDITIONAL READING MATERIAL


Closed test procedure

• n null hypotheses H1, …, Hn

• Closed test procedure considers all intersection hypotheses.

• Hi is rejected at global level ifall hypotheses HI formed by intersection with Hi arerejected at local level

H1 can only be rejected

at =.05 if H12 is also

rejected at =.05

Source: Jeff Maca

CLOSED TESTING


• A typical study with 3 doses 3 pairwise hypotheses.

• Multiplicity can be handled by adjusting p-values from each stage using Simes procedure

iSi

S pi

Sq min S is number of elements in Hypothesis,

p(i) is the ordered P-values

Source: Jeff Maca

CLOSED TESTING (cont.)


Stage sample sizes: n1 = 75, n2 =75

Unadjusted pairwise p-values from the first stage:

p1,1= 0.23, p1,2 = 0.18, p1,3 = 0.08

Dose 3 selected at interim

Unadjusted p-value from second stage: p2,3 = .01

Source: Jeff Maca

SCENARIO: DOSE FINDING

3 DOSES & CONTROL


q1,123 = min( 3*.08, 1.5*.18, 1*. 23)= .23

q2,123 = p2,3 = .01

C(q1,123, q2,123) = 2.17 P value = .015

Source: Jeff Maca

THREE-WAY TEST


q1,13 = min( 2*.08, 1*. 23)= .16

q1,23 = min(2*.08,1*.18) = .16

q2,13 = q2,23 = p2,3 = .01

C(q1,13, q2,13) = C(q1,23, q2,23) = 2.35 P.value = .0094

Source: Jeff Maca

TWO-WAY TEST


q1,3 = p1,3 = .08

q2,3 = p2,3 = .01

C(q1,3, q2,3) = 2.64 P.value = .0042

Conclusion: Dose 3 is effective

Source: Jeff Maca

FINAL TEST


SOFTWARE IMPLEMENTATIONStable, documented software is available for the CRM and its variations. Can be downloaded from:

• http://www.cancerbiostats.onc.jhmi.edu/Software.cfm (Johns Hopkins Comprehensive Cancer Center)

• http://biostatistics.mdanderson.org/SoftwareDownload (M.D. Anderson)

Computations for other adaptive dose finding methods, such as the NDLM and D-Optimal approaches, are more complex and intensive – software implementation is often based on customized code (e.g., Fortran programs)

Because of computational intensity, generally use lower level programming language (e.g., C++, Fortran) – R and SAS implementations would be too slow


SOFTWARE IMPLEMENTATION (CONT.)

Commercial software currently in prototype phase and under testing – not yet available for general use.

• GADA implementations of NDLM (Tessella)

• Bayesian D-Optimal and variations (CytelSim)

Different pharma companies are developing their own adaptive dose finding software, in combination with simulation software for evaluation of Op. Characteristics

Opportunity for pre-competitive collaboration via PhRMAto develop more general purpose software – possible consortium sponsored by PhRMA member companies


Practical Considerations


PRACTICAL CONSIDERATIONS

Assessing projects for an adaptive design:

• Rapid acquisition of data relative to enrollment rate

- Outcome is more immediate (and accurately) observable

- Trials of longer duration, with relatively slow recruitment, canbe good candidates

• Existence of predictive biological models and/or prior information in patient population of interest

- Predictive models for longer term outcome



Assessing projects for an adaptive design (cont.):

• Ethical considerations as driver for adaptation

• A high exploratory aspect may indicate greater efficacy gains

- Uncertainty relative to e.g., dose, variability, effect size

- Wide dose range

• Caution: Assuming the patient population remains constant over time

- Trials of long duration

- Selection bias due to unblinding of information



Considerations for simulations:

• Leverage information from disease state and exposure-response models

- Selecting dose-response model

- Defining prior distributions for model parameters

- Development of adaptive algorithm (decision criteria)

- Trial simulations to assess design performance

• Optimize over PD response model (best guess of truth)

• Assess sensitivity to using other response models (include models different from the design dose-response model)



Considerations for simulations (cont.):

• Understand impact of enrollment rate

- Include different rates in simulations

- Consider controlling rate if simulations indicate gains

• Assess impact across different dropout models

• Include information lag (e.g. batches) in simulations

• Demonstrating control of Type I error rate

- Simulate over a grid of scenarios in the null space

- Simulate across various dropout & enrollment rate models



Practical Issues for implementation:

• Additional time to develop the design & protocol

- May need to run extensive simulations to understand operating characteristics

- Communicate with the primary investigator’s about the design, receive feedback, and address concerns

• Clinical trial material needs

- Dosage strengths, quantity, packaging

• Additional resources for modeling & data analysis

- Interim data preparations & analyses

- Final analysis more complex



Practical Issues for implementation (cont.):

• Increase in site communication

- Design changes / Patient treatment assignments

- Fax, Interactive Voice Response Systems, WEB interface

- Additional site training

• Determine type of committee needed to monitor trial

- Ensure protocol is followed (no programming errors)

- Unanticipated safety signals not accounted for in the adaptive algorithm

- Engage committee early in scenario simulations (prior to protocol approval)



Practical Issues for implementation (cont.):

• Determine what data will be needed

• How data will be collected?

- Electronic data capture, Expedited report forms (not monitored), Voice Response system, Excel Spreadsheet

- eDC systems not friendly for interim data extraction

• How clean data needs to be?

- Fully verified (locked) data is not typical

- Use latest data in modeling/analysis (continually clean data)

• Document, Document, Document!!!

Finding Studies Adaptive Designs in Dose- · 2007. 11. 14. · Webinar 2: Adaptive designs in...

Documents

Transcript of Finding Studies Adaptive Designs in Dose- · 2007. 11. 14. · Webinar 2: Adaptive designs in...