Post on 25-Feb-2016
description
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
1
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Alpha Recyclingin
Confirmatory Clinical TrialsUnifies, Simplifies, Extends
many common MTPs
Olivier GuilbaudSenior Principal Scientist
AstraZeneca R&D, Södertälje, Sweden
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
2
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Outline
• Background
• Splitting, Recycling, Adding, (parts of alpha)
• Graphs and Default Graphs for MTPs
• Rejection Algorithm
• Holm’s MTP for groups of Hs, and a problem to discuss
• Improvements through extensions of the Default Graph
• Summary, and further results:
• (Simultaneous Confidence regions, CTP-formulation of MTPs)
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
3
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Background: Regulatory environment
Citation from EMEA/CPMP’s (2002) Points to Consider on Multiplicity Issues in Clinical Trials :
“ … multiplicity can have a substantial influence on the rate of false positive conclusions …”
”As a general rule it can be stated that control of the family-wise type-I error rate in the strong sense … is a minimal prerequisite for confirmatory claims”
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
4
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Background: Multiple Confirmatory Comparisons
• Several Efficacy Variables • Several Tolerability Variables • Several Comparators (e.g. Placebo, Active 1, Active 2,
…)• Several Doses
• Delta-Noninferiority / Superiority / Delta-Superiority • Several Subgroups/Kinds of Subjects
• Several kinds of Administration (e.g. once daily, twice daily, …)
Confirmatory:Pre-specified family of multiple comparisons (i.e. statistical tests) .Risk of getting any ”false positive” result/conclusion is (no matter how many or which H0s are true – Strong Control)
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
5
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Background: Abbreviations/Terminology
• MTP : Multiple Testing Procedure
• MCP : Multiple Comparison Procedure
• FWER : Family-Wise Error Rate (type-I errors) =Pr[at least 1 true Hi in the Family is rejected by the
MTP](to be controlled to be in the strong sense)
, no matter how many, or which, His in the family are true
In contrast to the weak sense (old) : , if all His in the family are true
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
6
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Background: brief refresher about three basic MTPs
Three basic MTPs for Family {H1, H2, H3} based on raw p-values p1, p2, p3
• Holm MTP (1979)p(1) /3, p(2) /2, p(3) /1
• Fixed-Sequence MTP (old) p1 , p2 , p3
• Fallback MTP (2003/2005)weights w1, w2, w3 p1 w1, p2 (w2+), p3 (w3+)
Now, to the Alpha-Recycling framework
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
7
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Two articles in Statistics in Medicine, February 2009
Only this now
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
8
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Idea: Splitting and Recycling test-mass (= part of )
Bonferroni:
split of Fixed-Sequence: recycling of Combination of
split & recycling of
Throughout: Raw p-value available for each null hypothesis H
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
9
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Idea: Adding test-mass from different paths
Parallel-gatekeeping MTP: 2 graphs for same MTP
Default graph
Important for rejection AlgorithmIts sequences reflect the possible rejection paths
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
10
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Idea: Adding test-mass from different paths
Fallback MTP for 3 Hs: 2 graphs for same MTP
Default graph
Important for rejection AlgorithmIts sequences reflect the possible rejection paths
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
11
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Idea: Adding test-mass from different paths
Holm’s MTP for 3 Hs: 2 graphs for same MTP
Default graph
Important for rejection AlgorithmIts sequences reflect the possible rejection paths
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
12
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Rejection Algorithm - based on Default-Graph Seqs
This MTP controls (in the strong sense) the FWER to be !
Step 1. For each ”first” H in Sequences,
(a) add-upp test mass from ”its” Sequences(b) test H at this added-up level
If nothing is rejected, then Stop; otherwise:
Reduce Sequences by deleting all rejected H from Sequences, and go to Step 2 with remaining reduced Sequences
Step r = 2, 3, …. For each ”first” H in remaining reduced Sequences,
(a) add-upp test mass from ”its” Sequences(b) test H at this added-up level
If nothing is rejected, then Stop; otherwise:
Reduce Sequences by deleting all rejected H from Sequences, and go to Step r + 1 with remaining reduced Sequences
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
13
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Comments
• number of sequences can be increased(no restriction)
• length of sequences can be increased(no restriction)
• if not all sequences contain all Hs, thenimprovements are possible (by increasing length of relevant sequences)
• interpretability of MTP is important(also when improvements are considered)
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
14
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
A refresher – Bonferroni & Holm for groups of Hs
Step1: Bonferroni-version as above
Step2: If one group is entirely rejected, but not the other, then try again
in the other at increased level
1 2
3 4
Bonferroni for 2 groups of 2 Hs, with
Fixed-Seq testing within group
/2 /2
Holm for 2 groups of 2 Hs, with Fixed-Seq testing within group:
1 2
3 4
/2 /2
1
1 2
3 4
/2
1
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
15
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Problem to discuss with your neighbour
1 2
3 4
Bonferroni for 2 groups of 2 Hs, with
Fixed-Seq testing within group
/2 /2
(is already in default-graph form):
Extend sequences of default graph to get recycling improvement equivalent toHolm for 2 groups of 2 Hs, with Fixed-Seq testing within group
1 2
3 4
/2 /2
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
16
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Recent Example discussed by FDA statisticians
• Confirmatory family with 4 null hypotheses :
H1 = (Null hypothesis concerning non-inferiority for primary variable) H2 = (Null hypothesis concerning superiority for primary variable) H3 = (Null hypothesis concerning non-inferiority for secondary variable) H4 = (Null hypothesis concerning superiority for secondary variable)
H1
H2 H3
H4
w (1-w)
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
17
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Recent Example discussed by FDA statisticians
Alpha-recycling applied to show Non-inf & Superiority for Primary & Secondary Variables
H1
H2 H3
H4
w (1-w)
Basic version
H1
H2 H3
w (1-w)
H1
H4
Reformulation in Recycling terms
Very easy to test with rejection algorithm !
But why not recycle also after the last H in each sequence ?
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
18
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Recent Example discussed by FDA statisticians
Alpha-recycling applied to show Non-inf & Superiority for Primary & Secondary Variables - Simple improvement
Improvement Reformulation in Recycling terms
H1
H2 H3
w (1-w)
H1
H4H3
H2H4
H1
H2 H3
H4
w (1-w)
Very easy to test with rejection algorithm !This improved MTP is ”-exhaustive” !
May be questionedby some (not me)!
May be questionedby some (not me)!
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
19
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Sequences can be extended/added for more rejections
Improved Fallback MTP for 3 Hs: 2 graphs for same MTP
Default graph
This improved MTP is ”-exhaustive” !
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
20
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Sequences can be extended/added for more rejections
Improved Parallel-gatekeeping : 2 graphs for same MTP
Default graph
This improved MTP is ”-exhaustive” !
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
21
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
XXX example: 2 primary & 2 second vars, 2 age stata
1
2 3
4
5 6
8
9 11
7
10 12
Original proposal
Simple improvement possible ( Holm instead of Bonf after H8 ):
Add sub-seq (11, 12) to Default-graph sequences ending with (9, 10) Add sub-seq (9, 10) to Default-graph sequences ending with (11, 12)
1
2
7
8
9
10
/81
3
7
8
9
10
/84
5
7
8
9
10
/84
6
7
8
9
10
/81
2
7
8
11
12
/81
3
7
8
11
12
/84
5
7
8
11
12
/84
6
7
8
11
12
/8
Default graphReformulation in Recycling terms
to show strong control of FWER
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
22
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Compact notation for MTP in XXX example
1
2 3
4
5 6
8
9 11
7
10 12
Original proposal
Compact notation for this recycling MTP (B here is Bonf-operator):
B(H1 B(H2, H3), H4 B(H5, H6)) H7 H8 B(H9 H10, H11 H12)
Can be expanded (using certain rules) to 8 sequences of Hs above
1
2
7
8
9
10
/81
3
7
8
9
10
/84
5
7
8
9
10
/84
6
7
8
9
10
/81
2
7
8
11
12
/81
3
7
8
11
12
/84
5
7
8
11
12
/84
6
7
8
11
12
/8
Default graphReformulation in Recycling terms
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
23
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
YYY example: 3 doses vs. 0, 1 primary & 1 second var
1
3
1
5
2
5
2
4
6 6
/4 /4 /4 /4
Reformulation in Recycling termsDefault graph
1 2
3 45
6
Basic version
/2 /2
Idea: To give lowest dose a chance if at least 1 of the 2 larger doses works for the primary variable
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
24
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
YYY example – Improvement 1
1
3
1
5
2
5
2
4
6 6
/4 /4 /4 /4
Reformulation in Recycling termsDefault graph
5
6
5
6
1 2
3 45
6
Improvement 1
/2 /2
Increases level for H5 and H6 if H3 and/or H4 are rejected
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
25
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
YYY example – Improvement 2
1 2
3 45
6
Improvement 2
1/2 1/2
Gives H1 or H2 a (Holm-type) second opportunity to be rejected if not rejected initially
1
3
1
5
2
5
2
4
6 6
/4 /8 /8 /4
Reformulation in Recycling termsDefault graph
5
6
5
6
1
5
6
/8
2
5
6
/8
2
4
2 2
4
1 1
3 1
3
3
4 3
4
3 4
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
26
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Summary, and further results
• Easy to formulate (cf. Closed-Testing Pocedures) • Easy to perform (even manually)
• Easy to calculate multiplicity-adjusted p-values (algorithm similar to that for rejections)
• Easy to obtain weights of Bonferroni test for any intersection hypothesis HI of corresponding CTP(useful for contruction of Conf regions)
• Easy to construct new MTPs (for new problems)
• Unifies many common MTPs
• Compact algebraic notation available(to describe and derive default graphs)
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
27
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Corresponding Simultaneous Confidence Regions ???
• Holm MTP (1979)p(1) /3, p(2) /2, p(3) /1
• Fixed-Sequence MTP (old) p1 , p2 , p3
• Fallback MTP (2003/2005)weights w1, w2, w3 p1 w1, p2 (w2+), p3 (w3+)
Solution inJASA 1999
Open until2007
Three basic MTPs for Family {H1, H2, H3} based on raw p-values p1, p2, p3
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
28
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Corresponding Simultaneous Confidence Regions !!!
Amazing:
Simultanouslypresented at MCP 2007in Vienna
28 years after Holm (1979)
ExtensionsandRelations
Confidence regions require weights that can be obtained directly from rejection algorithm of recycling approach
Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY
34
O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29
Outline
• Background
• Splitting, Recycling, Adding, (parts of alpha)
• Graphs and Default Graphs for MTPs
• Rejection Algorithm
• Holm’s MTP for groups of Hs, and a problem to discuss
• Improvements through extensions of the Default Graph
• Summary, and further results:
• (Simultaneous Confidence regions, CTP-formulation of MTPs)