OutlineIntroduction
ResultsImprovementFuture plans
Multiple Testing in Group Sequential Clinical Trials
Tian Zhao
Supervisor: Michael Baron
Department of Mathematical SciencesUniversity of Texas at Dallas
7/20/2013
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
1 IntroductionSequential statisticsProblem
2 ResultsBonferroni approach
3 ImprovementHolm-type stepwise methods
Truncated Sequential Probability Ratio Test
Fallback approach
4 Future plans
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Sequential statistics
Multiple testing in clinical trials
Multiple endpoints ( efficacy and safety )Multiple treatment arms or doses of a drugOverall, get an answer to several questions
Sequential / group sequential clinical trials
Interim analysis (Flector trial: stop for futility)Sequential clinical trials always have a restriction on themaximum number of sampled groups (K)
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Sequential statistics
Multiple testing in clinical trials
Multiple endpoints ( efficacy and safety )Multiple treatment arms or doses of a drugOverall, get an answer to several questions
Sequential / group sequential clinical trials
Interim analysis (Flector trial: stop for futility)Sequential clinical trials always have a restriction on themaximum number of sampled groups (K)
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Our problem
Test multiple hypotheses based on sequentially observed data,achieving a decision on each individual test
Group sequential sampling
Formulation
Goal
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Our problem
Test multiple hypotheses based on sequentially observed data,achieving a decision on each individual test
Group sequential sampling
Formulation
Goal
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Our problem
Test multiple hypotheses based on sequentially observed data,achieving a decision on each individual test
Group sequential sampling
Formulation
Goal
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Group sequential methods
Group sequential sampling
A number of group sequential procedures have been proposedfor testing ONE hypothesis.
Pocock (1977) applies repeated significance tests to groupsequential trials with equal-size groups and derives a constantcritical value on the standardized normal Z scale across allstages that control Type I error. O’Brien and Fleming (1979)propose a sequential procedure that has boundary valuesdecrease over the stages on the standardized normal Z scale
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Formulation
Observe a sequence of iid random vectors Xn, n = 1, 2, ...
Xn = (X(1)1 , ...X
(d)n ) ∈ Rd is a set of measurements on unit n.
X(j)i ∼ fj(x |θ(j))
Multiple hypotheses
Test d hypotheses:
H(j)0 : θ(j) = θ
(j)0 vs H
(j)A : θ(j) = θ
(j)1 , for j = 1, 2, ..., d
Let m = constant be the group size and T be a stopping time
Tests are based on a group sequential experiment. Data arecollected in groups of m patients.
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Sequential statisticsProblem
Goal
ControlFamilywise Type I error rate = FWER-I = P( At least oneType I error ) ≤ αFamilywise Type II error rate = FWER-II = P( At least oneType II error) ≤ βP(T ≤ K ) = 1 , T is stopping time, K = given integer =max allowed number of groups
Do it at low expected cost
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Bonferroni approach
Bonferroni approach
Bonferroni approach : control of both error rates.Choose
αj ∈ (0, 1),∑
αj = α; βj ∈ (0, 1),∑
βj = β
Test H(j)0 at level αj and power (1− βj) by any known sequential
procedure
P
d⋃j=1
Type I error on H(j)0
≤∑P(Type I error on H
(j)0
)=∑
αj = α
P
d⋃j=1
Type II error on H(j)A
≤∑P(Type II error on H
(j)A
)=∑
βj = β
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Improvement of Bonferroni Methods
Bonferroni inequality is very crude for moderate to large d, extendto Holm’s stepwise approach.
Order the log-likelihood ratio (LLR) statistics in decreasing(step-down)/increasing (step-up)
Choose stopping boundaries for the ordered LLR
Define the stopping rule and decision rules
This requires analysis of the group sequential test
Boundaries of Pocock, O’Brien-Fleming, Wang-Tsiatis testsare computed numerically, not tractable to analyze
Derive equations for Whitehead triangular test and TruncatedSPRT
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Group sequential tests
-
-
-
-
6
6
6
6
k
k
k
k
Pocock testZk O’Brien-Fleming testZk
Whitehead testZk
������
��
��������
Truncated SPRTΛk
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Derive Truncated SPRT
SPRT
Choose stopping boundaries a, b to control α and βwhere,
a =1− βα
b =β
1− αSPRT control both α, β among all sequential tests that have
P(Type I error) ≤ αP(Type II error) ≤ β
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Wald’s SPRT
6
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@
���������������������������
0
b
a
Λk
k
R e j e c t H0
A c c e p t H0
t���tXXXt���t���t���t
HHHt���t���t���u
T
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT
Stopping time and Decision rule
Following SPRT approach for single hypothesis testing, truncatedSPRT is based on log-likelihood rations
Λn = lnf (x |θ1)
f (x |θ0)
Stopping time
T = min(K ,min {n : Λn /∈ (b, a)}) ≤ K
Decision rule,
δ =
{reject H0 if ΛT ≥ a or ΛT ≥ c ∩ T = Kaccept H0 if ΛT ≤ b or ΛT < c ∩ T = K
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
6
-
@@@@@@@@@@@@@@@@@@@
�������������������
@@@@@@@@@
�����
0
b
a
Λk
k
c
R e j e c t H0
A c c e p t H0
K
Accept
H0
RejectH0
t���tXXXt���t���t���t
HHHt���u
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
Plan
Evaluate performance of TSPRT, probabilities of Type I andType II errors.
Find equations of the stopping boundaries and the requiredgroup size to control P{Type I error} and P{Type II error}.Then develop Holm-type stepwise sequential procedures totest d hypotheses controlling familywise error rates I and II.
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT. Performance evaluation.
Evaluate probabilities of Type I and Type II errors by comparisonbetween TSPRT and SPPRT.
6
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@
���������������������������
0
b
a
Λk
k
c
R e j e c t H0
A c c e p t H0
Kt���tXXXt���
t���t���tHHHt������t���
uu
Both tests reject H0
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT. Performance evaluation.
6
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@
���������������������������
0
b
a
Λk
k
c
R e j e c t H0
A c c e p t H0
Kt���tXXXt���
t���t���tHHHt���@
@@tAAAAAAu
uTSPRT rejects H0
but SPRT accepts it
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT. Performance evaluation.
6
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@
���������������������������
0
b
a
Λk
k
c
R e j e c t H0
A c c e p t H0
Kt���tXXXt���
t���tHHHt tXXX@@@@@@
tu
u
Both tests accept H0
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT. Performance evaluation.
6
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@
���������������������������
0
b
a
Λk
k
c
R e j e c t H0
A c c e p t H0
Kt���tXXXt���
t���tHHHt tXXX������tu
u
TSPRT accepts H0
but SPRT rejects it
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Probability of Type I Error
Choose c to attain P(Type I error |TSPRT ) ≤ α.
Let ΛK =∑Km
j=1 lnf (xj |θ1)f (xj |θ0) . For a given α1=level for SPRT,
P(Type I error |TSPRT )
= PH0(Type I error |SPRT )
+PH0({Λ1, ...ΛK−1 ∈(b, a)} ∩ {ΛK ∈ [c , a)} ∩ {Λn ≤ b before Λn ≥ a})−PH0({Λ1, ...ΛK−1 ∈(b, a)} ∩ {ΛK ∈(b, c)} ∩ {Λn ≥ a before Λn ≤ b})≤ α1 + P(A)− P(B)
We need to make P(A) ≤ α− α1
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Type I error
P(A) ≤ PH0(ΛK ∈ [c, a))PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)
≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)
≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK = c)
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
Derivations
Assumption: ( General ) Exponential familyTest
H0 : θ = θ0 vs HA : θ = θ1.
Exponential family is written in the canonical form withη = canonical parameter as follows,
f (x |η) = f (x |0)eηx−ψ(η)
where, ψ(0) = 0 and µ(η) = EX = ψ′(η), VarX = ψ”(η)
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
Chernoff Inequality
In general case, we have
P(n∑
i=1
Zj ≥ ny) ≤ e−n supt≥0{ty−ψz (t)}
here,
Z = lnf (x |θ1)
f (x |θ0), c = ny
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
Normal as example
Observe X1,X2, ...,Xn ∼ N(θ, σ2), we can write it as
f (x |θ, σ2) =1√2πσ
e−x2
2σ2 eθxσ2−
θ2
2σ2
here,
η =θ
σ2, f (x |0) =
1√2πσ
e−x2
2σ2 , ψx(η) =σ2η2
2
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Truncated SPRT (cont’d)
Normal as example
By Chernoff Inequality,
supt≥0{ty − ψz(t)} attained at t̂ = [ψ′z ]−1(y)
Therefore,
P(n∑
i=1
Zj ≥ ny) ≤ e−n{ σ2y2
2(θ1−θ0)2+
12y+
(θ1−θ0)2
8σ2 }
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Type I error
P(A) ≤ PH0(ΛK ∈ [c, a))PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)
≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK ≥ c)
≤ PH0(ΛK ≥ c)PH0(Λn ≤ b before Λn ≥ a|ΛK = c)
Hence, we have
P(A) ≤ 1− α1ec
1− α1βe−n{ σ2c2
2n2(θ1−θ0)2+
c2n+
(θ1−θ0)2
8σ2 } ≤ α− α1
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Type II error
For α1=level for SPRT ∈ (0, α) , β1= power for SPRT ∈ (0, β)Let a = − lnα1, and b = − lnβ1
Choose c1 to control Type I error:
1− α1ec1
1− α1β1e−n{ σ2c21
2n2(θ1−θ0)2+
c12n+
(θ1−θ0)2
8σ2 } ≤ α− α1
Choose c2 to control Type II error:
1− β1ec21− α1β1
e−n{ σ2c22
2n2(θ1−θ0)2−
c22n+
(θ1−θ0)2
8σ2 } ≤ β − β1
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Type II error
Therefore, if θ1−θ0σ > 0, we have,
c1 ≥
(√−2
nln (α− α1)(1− α1β1)− (θ1 − θ0)
2σ
)n(θ1 − θ0)
σ
c2 ≤
(−√−2
nln (β − β1)(1− α1β1) +
(θ1 − θ0)
2σ
)n(θ1 − θ0)
σ
Minimize n to control both Type I error and Type II error
n∗ =2σ2
(√− ln (α− α1)(1− α1β1) +
√− ln (β − β1)(1− α1β1)
)2(θ1 − θ0)2
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Stepwise method for Truncated SPRT
Test d hypotheses:
H(j)0 : θ(j) = θ
(j)0 vs H
(j)A : θ(j) = θ
(j)1 j = 1, 2, ..., d
Test statistics Λ(j)k =
∑ln
fj (X(j)i |θ
(j)1 )
fj (X(j)i |θ
(j)0 )
Order test statistics Λ[1]k ≥ ... ≥ Λ
[d ]k , let H
(j)0 be the
corresponding tested null hypotheses arranged in the sameorder.
aj = − ln α−α1d−j+1 , bj = ln β−β1
j
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Stepwise method for Truncated SPRT(cont’d)
Decision rule (reject one by one, accept all)
If Λ[1]k ≤ b1 ⇒ stop, accept H
[1]0 , ...,H
[d ]0
If Λ[1]k ≥ a1 ⇒ reject H
[1]0 , go to H
[2]0
If Λ[1]k ∈ (b1, a1), ⇒ continue sampling
......
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Stepwise method for Truncated SPRT(cont’d)
At k = K ,
If Λ[j]K /∈ (bj , aj), ⇒ stop
If Λ[j]K ∈ (bj , aj),
Λ[j]K > cj ⇒ stop, reject
Λ[j]K < cj ⇒ stop, accept
This stepwise method can control FWER-I and FWER-II
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Holm-type stepwise methodsFallback approach
Fallback approach
Pocock Test
Pocock derives the constant boundary [−Cp,Cp] on thestandardized Z scale, if Z /∈ [−Cp,Cp], stop and reject H0,otherwise, keep sampling, after K groups, if Z /∈ [−Cp,Cp], stopand reject H0, otherwise, stop and accept H0
Fallback method for Pocock: recycle βj on all H(j)0 that were
rejected, then at k = K , test remaining H(j)0 at higher βj
βj(K ) =βj(1)
1−∑
i :H(i)0 was rejected
βi
Testing with higher β means that we can accept more often,expand the acceptance region. P(reject|H0) reduces, we canshrink boundary [−Cp,Cp] to meet the α level, and then reduce m.
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Future plans
Use stepwise approach to multiple hypotheses starting fromvarious standard group sequential methods
Evaluate performance of fallback procedures, compare fallbackand stepwise testing approaches
Composite hypotheses and nuisance parameters. Sequentialt-test.
Examples of clinical data
Thank you!
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Future plans
Use stepwise approach to multiple hypotheses starting fromvarious standard group sequential methods
Evaluate performance of fallback procedures, compare fallbackand stepwise testing approaches
Composite hypotheses and nuisance parameters. Sequentialt-test.
Examples of clinical data
Thank you!
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Future plans
Use stepwise approach to multiple hypotheses starting fromvarious standard group sequential methods
Evaluate performance of fallback procedures, compare fallbackand stepwise testing approaches
Composite hypotheses and nuisance parameters. Sequentialt-test.
Examples of clinical data
Thank you!
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
OutlineIntroduction
ResultsImprovementFuture plans
Future plans
Use stepwise approach to multiple hypotheses starting fromvarious standard group sequential methods
Evaluate performance of fallback procedures, compare fallbackand stepwise testing approaches
Composite hypotheses and nuisance parameters. Sequentialt-test.
Examples of clinical data
Thank you!
Tian Zhao Supervisor: Michael Baron Multiple Testing in Group Sequential Clinical Trials
Top Related