Multiple Testing in Group Sequential Clinical Trials · Multiple testing in clinical trials...

OutlineIntroduction

ResultsImprovementFuture plans

Multiple Testing in Group Sequential Clinical Trials

Tian Zhao

Supervisor: Michael Baron

Department of Mathematical SciencesUniversity of Texas at Dallas

[email protected]

7/20/2013

Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials

OutlineIntroduction


1 IntroductionSequential statisticsProblem

2 ResultsBonferroni approach

3 ImprovementHolm-type stepwise methods

Truncated Sequential Probability Ratio Test

Fallback approach

4 Future plans


OutlineIntroduction


Sequential statisticsProblem

Sequential statistics

Multiple testing in clinical trials

Multiple endpoints ( efficacy and safety )Multiple treatment arms or doses of a drugOverall, get an answer to several questions

Sequential / group sequential clinical trials

Interim analysis (Flector trial: stop for futility)Sequential clinical trials always have a restriction on themaximum number of sampled groups (K)


OutlineIntroduction



Our problem

Test multiple hypotheses based on sequentially observed data,achieving a decision on each individual test

Group sequential sampling

Formulation

Goal


OutlineIntroduction



Group sequential methods

Group sequential sampling

A number of group sequential procedures have been proposedfor testing ONE hypothesis.

Pocock (1977) applies repeated significance tests to groupsequential trials with equal-size groups and derives a constantcritical value on the standardized normal Z scale across allstages that control Type I error. O’Brien and Fleming (1979)propose a sequential procedure that has boundary valuesdecrease over the stages on the standardized normal Z scale


OutlineIntroduction



Formulation

Observe a sequence of iid random vectors Xn, n = 1, 2, ...

Xn = (X(1)1 , ...X

(d)n ) ∈ Rd is a set of measurements on unit n.

X(j)i ∼ fj(x |θ(j))

Multiple hypotheses

Test d hypotheses:

H(j)0 : θ(j) = θ

(j)0 vs H

(j)A : θ(j) = θ

(j)1 , for j = 1, 2, ..., d

Let m = constant be the group size and T be a stopping time

Tests are based on a group sequential experiment. Data arecollected in groups of m patients.


OutlineIntroduction



Goal

ControlFamilywise Type I error rate = FWER-I = P( At least oneType I error ) ≤ αFamilywise Type II error rate = FWER-II = P( At least oneType II error) ≤ βP(T ≤ K ) = 1 , T is stopping time, K = given integer =max allowed number of groups

Do it at low expected cost


OutlineIntroduction


Bonferroni approach

Bonferroni approach

Bonferroni approach : control of both error rates.Choose

αj ∈ (0, 1),∑

αj = α; βj ∈ (0, 1),∑

βj = β

Test H(j)0 at level αj and power (1− βj) by any known sequential

procedure

P

d⋃j=1

Type I error on H(j)0

≤∑P(Type I error on H

(j)0

)=∑

αj = α

P

d⋃j=1

Type II error on H(j)A

≤∑P(Type II error on H

(j)A

)=∑

βj = β


OutlineIntroduction


Holm-type stepwise methodsFallback approach

Improvement of Bonferroni Methods

Bonferroni inequality is very crude for moderate to large d, extendto Holm’s stepwise approach.

Order the log-likelihood ratio (LLR) statistics in decreasing(step-down)/increasing (step-up)

Choose stopping boundaries for the ordered LLR

Define the stopping rule and decision rules

This requires analysis of the group sequential test

Boundaries of Pocock, O’Brien-Fleming, Wang-Tsiatis testsare computed numerically, not tractable to analyze

Derive equations for Whitehead triangular test and TruncatedSPRT


OutlineIntroduction



Group sequential tests

-

-

-

-

6

6

6

6

k

k

k

k

Pocock testZk O’Brien-Fleming testZk

Whitehead testZk

��

��

��

Truncated SPRTΛk


OutlineIntroduction



Derive Truncated SPRT

SPRT

Choose stopping boundaries a, b to control α and βwhere,

a =1− βα

b =β

1− αSPRT control both α, β among all sequential tests that have

P(Type I error) ≤ αP(Type II error) ≤ β


OutlineIntroduction



Wald’s SPRT

6

-

@@@@@@@@@@@@@@@@@@@@@@@@@@@

��

0

b

a

Λk

k

R e j e c t H0

A c c e p t H0

t��tXXXt��t��t��t

HHHt��t��t��u

T


OutlineIntroduction



Truncated SPRT

Stopping time and Decision rule

Following SPRT approach for single hypothesis testing, truncatedSPRT is based on log-likelihood rations

Λn = lnf (x |θ1)

f (x |θ0)

Stopping time

T = min(K ,min {n : Λn /∈ (b, a)}) ≤ K

Decision rule,

δ =

{reject H0 if ΛT ≥ a or ΛT ≥ c ∩ T = Kaccept H0 if ΛT ≤ b or ΛT < c ∩ T = K


OutlineIntroduction



Truncated SPRT (cont’d)

6

-

@@@@@@@@@@@@@@@@@@@

��

@@@@@@@@@

��

0

b

a

Λk

k

c

R e j e c t H0

A c c e p t H0

K

Accept

H0

RejectH0

t��tXXXt��t��t��t

HHHt��u


OutlineIntroduction




Plan

Evaluate performance of TSPRT, probabilities of Type I andType II errors.

Find equations of the stopping boundaries and the requiredgroup size to control P{Type I error} and P{Type II error}.Then develop Holm-type stepwise sequential procedures totest d hypotheses controlling familywise error rates I and II.


OutlineIntroduction



Truncated SPRT. Performance evaluation.

Evaluate probabilities of Type I and Type II errors by comparisonbetween TSPRT and SPPRT.

6

-

@@@@@@@@@@@@@@@@@@@@@@@@@@@

��

0

b

a

Λk

k

c

R e j e c t H0

A c c e p t H0

Kt��tXXXt��

t��t��tHHHt��t��

uu

Both tests reject H0


OutlineIntroduction




6

-

@@@@@@@@@@@@@@@@@@@@@@@@@@@

��

0

b

a

Λk

k

c

R e j e c t H0

A c c e p t H0

Kt��tXXXt��

t��t��tHHHt��@

@@tAAAAAAu

uTSPRT rejects H0

but SPRT accepts it


OutlineIntroduction




6

-

@@@@@@@@@@@@@@@@@@@@@@@@@@@

��

0

b

a

Λk

k

c

R e j e c t H0

A c c e p t H0

Kt��tXXXt��

t��tHHHt tXXX@@@@@@

tu

u

Both tests accept H0


OutlineIntroduction




6

-

@@@@@@@@@@@@@@@@@@@@@@@@@@@

��

0

b

a

Λk

k

c

R e j e c t H0

A c c e p t H0

Kt��tXXXt��

t��tHHHt tXXX��tu

u

TSPRT accepts H0

but SPRT rejects it


OutlineIntroduction



Probability of Type I Error

Choose c to attain P(Type I error |TSPRT ) ≤ α.

Let ΛK =∑Km

j=1 lnf (xj |θ1)f (xj |θ0) . For a given α1=level for SPRT,

P(Type I error |TSPRT )

= PH0(Type I error |SPRT )

+PH0({Λ1, ...ΛK−1 ∈(b, a)} ∩ {ΛK ∈ [c , a)} ∩ {Λn ≤ b before Λn ≥ a})−PH0({Λ1, ...ΛK−1 ∈(b, a)} ∩ {ΛK ∈(b, c)} ∩ {Λn ≥ a before Λn ≤ b})≤ α1 + P(A)− P(B)

We need to make P(A) ≤ α− α1


OutlineIntroduction



Type I error

P(A) ≤ PH0(ΛK ∈ [c, a))PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)

≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)

≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK = c)


OutlineIntroduction




Derivations

Assumption: ( General ) Exponential familyTest

H0 : θ = θ0 vs HA : θ = θ1.

Exponential family is written in the canonical form withη = canonical parameter as follows,

f (x |η) = f (x |0)eηx−ψ(η)

where, ψ(0) = 0 and µ(η) = EX = ψ′(η), VarX = ψ”(η)


OutlineIntroduction




Chernoff Inequality

In general case, we have

P(n∑

i=1

Zj ≥ ny) ≤ e−n supt≥0{ty−ψz (t)}

here,

Z = lnf (x |θ1)

f (x |θ0), c = ny


OutlineIntroduction




Normal as example

Observe X1,X2, ...,Xn ∼ N(θ, σ2), we can write it as

f (x |θ, σ2) =1√2πσ

e−x2

2σ2 eθxσ2−

θ2

2σ2

here,

η =θ

σ2, f (x |0) =

1√2πσ

e−x2

2σ2 , ψx(η) =σ2η2

2


OutlineIntroduction




Normal as example

By Chernoff Inequality,

supt≥0{ty − ψz(t)} attained at t̂ = [ψ′z ]−1(y)

Therefore,

P(n∑

i=1

Zj ≥ ny) ≤ e−n{ σ2y2

2(θ1−θ0)2+

12y+

(θ1−θ0)2

8σ2 }


OutlineIntroduction



Type I error

P(A) ≤ PH0(ΛK ∈ [c, a))PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)

≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)

≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK = c)

Hence, we have

P(A) ≤ 1− α1ec

1− α1βe−n{ σ2c2

2n2(θ1−θ0)2+

c2n+

(θ1−θ0)2

8σ2 } ≤ α− α1


OutlineIntroduction



Type II error

For α1=level for SPRT ∈ (0, α) , β1= power for SPRT ∈ (0, β)Let a = − lnα1, and b = − lnβ1

Choose c1 to control Type I error:

1− α1ec1

1− α1β1e−n{ σ2c21

2n2(θ1−θ0)2+

c12n+

(θ1−θ0)2

8σ2 } ≤ α− α1

Choose c2 to control Type II error:

1− β1ec21− α1β1

e−n{ σ2c22

2n2(θ1−θ0)2−

c22n+

(θ1−θ0)2

8σ2 } ≤ β − β1


OutlineIntroduction



Type II error

Therefore, if θ1−θ0σ > 0, we have,

c1 ≥

(√−2

nln (α− α1)(1− α1β1)− (θ1 − θ0)

2σ

)n(θ1 − θ0)

σ

c2 ≤

(−√−2

nln (β − β1)(1− α1β1) +

(θ1 − θ0)

2σ

)n(θ1 − θ0)

σ

Minimize n to control both Type I error and Type II error

n∗ =2σ2

(√− ln (α− α1)(1− α1β1) +

√− ln (β − β1)(1− α1β1)

)2(θ1 − θ0)2


OutlineIntroduction



Stepwise method for Truncated SPRT

Test d hypotheses:

H(j)0 : θ(j) = θ

(j)0 vs H

(j)A : θ(j) = θ

(j)1 j = 1, 2, ..., d

Test statistics Λ(j)k =

∑ln

fj (X(j)i |θ

(j)1 )

fj (X(j)i |θ

(j)0 )

Order test statistics Λ[1]k ≥ ... ≥ Λ

[d ]k , let H

(j)0 be the

corresponding tested null hypotheses arranged in the sameorder.

aj = − ln α−α1d−j+1 , bj = ln β−β1

j


OutlineIntroduction



Stepwise method for Truncated SPRT(cont’d)

Decision rule (reject one by one, accept all)

If Λ[1]k ≤ b1 ⇒ stop, accept H

[1]0 , ...,H

[d ]0

If Λ[1]k ≥ a1 ⇒ reject H

[1]0 , go to H

[2]0

If Λ[1]k ∈ (b1, a1), ⇒ continue sampling

......


OutlineIntroduction



Stepwise method for Truncated SPRT(cont’d)

At k = K ,

If Λ[j]K /∈ (bj , aj), ⇒ stop

If Λ[j]K ∈ (bj , aj),

Λ[j]K > cj ⇒ stop, reject

Λ[j]K < cj ⇒ stop, accept

This stepwise method can control FWER-I and FWER-II


OutlineIntroduction



Fallback approach

Pocock Test

Pocock derives the constant boundary [−Cp,Cp] on thestandardized Z scale, if Z /∈ [−Cp,Cp], stop and reject H0,otherwise, keep sampling, after K groups, if Z /∈ [−Cp,Cp], stopand reject H0, otherwise, stop and accept H0

Fallback method for Pocock: recycle βj on all H(j)0 that were

rejected, then at k = K , test remaining H(j)0 at higher βj

βj(K ) =βj(1)

1−∑

i :H(i)0 was rejected

βi

Testing with higher β means that we can accept more often,expand the acceptance region. P(reject|H0) reduces, we canshrink boundary [−Cp,Cp] to meet the α level, and then reduce m.


OutlineIntroduction


Future plans

Use stepwise approach to multiple hypotheses starting fromvarious standard group sequential methods

Evaluate performance of fallback procedures, compare fallbackand stepwise testing approaches

Composite hypotheses and nuisance parameters. Sequentialt-test.

Examples of clinical data

Thank you!


Multiple Testing in Group Sequential Clinical Trials · Multiple testing in clinical trials...

Documents

Transcript of Multiple Testing in Group Sequential Clinical Trials · Multiple testing in clinical trials...