Novel multiple testing procedures for structured study ...

Novel multiple testing proceduresfor structured study objectives

and families of hypotheses – a case study

Guenther Mueller-Velten

Novartis Pharma AG

EMA Workshop on Multiplicity Issues in Clinical TrialsLondon, 16 November 2012

Acknowledgement: Frank Bretz, Bjoern Holzhauer, Willi Maurer

1

Outline

• Introduction of graphical approaches to multiple testing• Case study

• Background and clinical considerations• Resulting multiple testing procedure• Interim analysis

• Summary and conclusions

2

Graphical approaches to multiple testingMotivation

Increasing complexity of confirmatory trial designs

• Multiple treatment arms, multiple primary and secondaryendpoints, interim analyses

• Designing a valid multiple testing strategy with desiredproperties is a cross-functional exercise and may involveseveral iterations

• Clinical, regulatory and business requirements need to betranslated into a statistical testing procedure in atransparent and understandable way

3

Graphical approaches to multiple testingMotivation (cont’d)

Graphical approaches provide a framework to

• Tailor advanced multiple test procedures to structuredfamilies of hypotheses

• Visualize complex decision strategies in an efficient andeasily communicable way, and

• Ensure strong Type I error rate control (Bretz et al., 2009;Burman et al., 2009)

4

Graphical approaches to multiple testingHeuristics

Notation

• Null hypotheses H1, . . . ,Hk

• Initial allocation of the significance level α = α1 + . . .+ αk .

• Unadjusted p-values p1, . . . ,pk

5

Graphical approaches to multiple testingHeuristics

Notation

• Null hypotheses H1, . . . ,Hk

• Initial allocation of the significance level α = α1 + . . .+ αk .

• Unadjusted p-values p1, . . . ,pk

“α propagation”

If a hypothesis Hi can be rejected at level αi (i.e. pi ≤ αi ),reallocate its level αi to the remaining, not yet rejectedhypotheses (according to a prefixed rule) and continue testingwith the resulting α levels.

6

Graphical approaches to multiple testingConventions

7


1 Hypotheses H1, . . . ,Hk

represented as nodesH1 H2

8



represented as nodes

2 Split of significance level αas weights α1, . . . , αk

H1 H2

H1 H2

α1 =α

2 α2 =α

2

9



represented as nodes

2 Split of significance level αas weights α1, . . . , αk

3 “α propagation” throughweighted, directed edges

H1 H2

H1 H2

α

2α

2

H1 H2

α

2α

21

1

10

Case studyIntroduction and background

• Randomized double-blind event driven outcome trial instable post myocardial infarction (MI) patients

• Three doses of a new therapy vs. placebo on top ofstandard of care

• No validated surrogate available for dose-finding prior toPhase III

• Primary endpoint is composite of CV death, MI or stroke• Two key secondary endpoints targeting additional label

claims, thus included in multiple testing procedure• Extended composite endpoint including hospitalization for

unstable angina requiring urgent unplannedrevascularizations

• New onset Type 2 diabetes among patients withpre-diabetes at baseline

• Additional multiplicity due to efficacy interim analyses11

Case studyClinical considerations

• Primary endpoint is essential to establish efficacy of therespective dose group. Key secondary objectives targetadditional label claims for doses that have establishedefficacy based on the primary endpoint.

• Successiveness: Do not reject a secondary hypothesiswithout having rejected the associated primary hypothesis.(Maurer et al., 2011)

• Benefit risk considerations: Higher doses potentially moreefficacious and lower doses generally safer

• Allow testing of lower doses regardless of efficacy at higherdoses.

• Unequal split of significance level.

12

Case studyStructured family of hypotheses

• Four-armed trial comparing• Three dose levels + standard of care• Placebo + standard-of-care

• Three endpoints• Primary endpoint: composite of CV death, MI or stroke• Two key secondary endpoints

⇒ Nine hypotheses• Three doses (low, medium, high)• Three endpoints per dose

13

Tailored multiple test procedure

high dose medium dose low dose

primary H1 H2 H3

14



primary

secondary

H1 H2 H3

H4 H5 H6

15



primary

secondary

H1 H2 H3

H4 H5 H6

The nodes for secondary endpoints represent families oftwo null hypotheses related to the two key secondary endpoints

16



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

17



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α

18



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α

19



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α

20



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α0.45 0.7

0.25

0.3 0.3 1

21



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α0.45 0.7

0.25

0.3 0.3 1

1 1 1

22



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α0.45 0.7

0.25

0.3 0.3 1

1 1 1

Key secondary endpoints within each dose group will be tested witha weighted Bonferroni-Holm test at the available local significance level

23



primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α0.45 0.7

0.25

0.3 0.3 1

1 1 1

Key secondary endpoints within each dose group will be tested witha weighted Bonferroni-Holm test at the available local significance level

Improvement of the multiple testing procedure by using weighted Dunnett’s test for allintersection hypotheses that contain at least two of H1,H2 and H3 for the same endpoint

24

Example rejection sequence


primary

secondary

H1 H2 H3

H4 H5 H6

0 0 0

0.2α 0.4α 0.4α

0.45 0.7

0.25

0.3 0.3 1

1 1 1

25



primary

secondary

H1 H2 H3

H4 H5 H6

0.06α 0 0

0.49α 0.45α

0.7

0.3 1

1 1 1

26



primary

secondary

H2 H3

H4 H5 H6

0 0

0.55α 0.45α

0.7

0.3 1

1 1

27



primary

secondary

H2 H3

H5 H6

0.165α 0

0.835α

1

1

0.3

0.7

28

Case study Interim analyses: Bonferroni inequality for the repeated hypothesis testing Interim analyses at 50% and 75% information fraction It

• A fixed Bonferroni split is used, with nominal overall significance levels α*t, t = 1, 2, 3, of 0.01% and 0.04% for the first and second efficacy interim analysis, leaving 2.45% for the final analysis.

• At each of the 3 analyses, the graphical testing procedure exploiting correlations between test statistics will be performed at the respective nominal level α*t , controlling the familywise type I error rate at the overall (one-sided) significance level α = 2.5%.

H1

H2

H9

It 0 50% 75% 100%

α*t = 0.01% 0.04% 2.45%

Note: If a primary hypothesis is rejected, the respective secondary hypothesis cannot be tested at the full level α, even if the trial is stopped !

29

Summary and conclusions

Confirmatory clinical trials are becoming increasingly more complex, often comparing multiple doses or treatments with a control for several primary and secondary endpoints.

The multiple study objectives are reflected by structured families of hypotheses that are characterized by multiple groups of “parent” primary hypotheses and “descendant” secondary hypotheses.

Novel graphical approaches for constructing and visualizing complex multiple test procedures with a focus on structured families of hypotheses are well suited to facilitate communication in clinical teams and to provide transparent decision strategies.

Graphical procedures ensure strong control of the overall Type I error rate across all primary and secondary hypotheses.

Multiple test procedure should be customized based on operating characteristics obtained via clinical scenario simulation.

Ideally, EMA could provide in its new guidance a harmonized terminology and framework categorizing study objectives and endpoints with respect to their impact on approval and labeling and respective need for type 1 error control.

30

References

Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009) A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine 28, 586-604.

Burman, C.-F., Sonesson, C. and Guilbaud, O. (2009) A recycling framework for the construction of Bonferroni-based multiple tests. Statistics in Medicine 28, 739–761.

Kordzakhia G. and Dmitrienko A. (2012) Superchain procedures in clinical trials with multiple objectives. Statist. Med. (early view)

Maurer, W., Glimm, E., and Bretz, F. (2011) Multiple and repeated testing of primary, co-primary and secondary hypotheses. Statistics in Biopharmaceutical Research 3, 336-352.

Rohmeyer, K., Klinglmueller, F., Bornkamp, B. (2012) gMCP: Graph Based Multiple Test Procedures. R package version 0.8-0. URL http://CRAN.R-project.org/package=gMCP

31

http://cran.r-project.org/package=gMCP



Back-up

32

Case study Improvement of the multiple testing procedure by using weighted Dunnett‘s test for all intersection hypotheses that contain at least two of H1, H2 and H3 for the same endpoint

0.45 0.63

First key secondary null hypotheses for high dose

0.27

First key secondary null hypotheses for medium dose

First Key secondary null hypotheses for low dose

0.27 0.72

0.4 × α 0.2 × α 0.4 × α

ε ε ε

0.25

0.07

0.07

0.13

Primary null hypothesis high dose

Primary null hypothesis medium dose

Primary null hypothesis low dose

0 0 0

Second key secondary null hypotheses for high dose

Second key secondary null hypotheses for medium dose

Second Key secondary null hypotheses for low dose

0.03 0.08

0.03

0 0 0

1 1 1 1-ε 1-ε 1-ε

33

Novel multiple testing procedures for structured study ...

Documents

Transcript of Novel multiple testing procedures for structured study ...