Novel multiple testing procedures for structured study objectives and families of hypotheses a case study

Size: px

Start display at page:

Download "Novel multiple testing procedures for structured study objectives and families of hypotheses a case study"

Kathlyn Walters
6 years ago
Views:

1 Novel multiple testing procedures for structured study objectives and families of hypotheses a case study Guenther Mueller-Velten Novartis Pharma AG EMA Workshop on Multiplicity Issues in Clinical Trials London, 16 November 2012 Acknowledgement: Frank Bretz, Bjoern Holzhauer, Willi Maurer 1

2 Outline Introduction of graphical approaches to multiple testing Case study Background and clinical considerations Resulting multiple testing procedure Interim analysis Summary and conclusions 2

3 Graphical approaches to multiple testing Motivation Increasing complexity of confirmatory trial designs Multiple treatment arms, multiple primary and secondary endpoints, interim analyses Designing a valid multiple testing strategy with desired properties is a cross-functional exercise and may involve several iterations Clinical, regulatory and business requirements need to be translated into a statistical testing procedure in a transparent and understandable way 3

4 Graphical approaches to multiple testing Motivation (cont d) Graphical approaches provide a framework to Tailor advanced multiple test procedures to structured families of hypotheses Visualize complex decision strategies in an efficient and easily communicable way, and Ensure strong Type I error rate control (Bretz et al., 2009; Burman et al., 2009) 4

5 Graphical approaches to multiple testing Heuristics Notation Null hypotheses H 1,..., H k Initial allocation of the significance level α = α α k. Unadjusted p-values p 1,..., p k 5

6 Graphical approaches to multiple testing Heuristics Notation Null hypotheses H 1,..., H k Initial allocation of the significance level α = α α k. Unadjusted p-values p 1,..., p k α propagation If a hypothesis H i can be rejected at level α i (i.e. p i α i ), reallocate its level α i to the remaining, not yet rejected hypotheses (according to a prefixed rule) and continue testing with the resulting α levels. 6

7 Graphical approaches to multiple testing Conventions 7

8 Graphical approaches to multiple testing Conventions 1 Hypotheses H 1,..., H k represented as nodes H 1 H 2 8

9 Graphical approaches to multiple testing Conventions 1 Hypotheses H 1,..., H k represented as nodes H 1 H 2 2 Split of significance level α as weights α 1,...,α k α 1 = α 2 α 2 = α 2 H 1 H 2 9

10 Graphical approaches to multiple testing Conventions 1 Hypotheses H 1,..., H k represented as nodes H 1 H 2 2 Split of significance level α as weights α 1,...,α k α 2 α 2 H 1 H 2 3 α propagation through weighted, directed edges α 2 1 α 2 H 1 H

11 Case study Introduction and background 11 Randomized double-blind event driven outcome trial in stable post myocardial infarction (MI) patients Three doses of a new therapy vs. placebo on top of standard of care No validated surrogate available for dose-finding prior to Phase III Primary endpoint is composite of CV death, MI or stroke Two key secondary endpoints targeting additional label claims, thus included in multiple testing procedure Extended composite endpoint including hospitalization for unstable angina requiring urgent unplanned revascularizations New onset Type 2 diabetes among patients with pre-diabetes at baseline Additional multiplicity due to efficacy interim analyses

12 Case study Clinical considerations Primary endpoint is essential to establish efficacy of the respective dose group. Key secondary objectives target additional label claims for doses that have established efficacy based on the primary endpoint. Successiveness: Do not reject a secondary hypothesis without having rejected the associated primary hypothesis. (Maurer et al., 2011) Benefit risk considerations: Higher doses potentially more efficacious and lower doses generally safer Allow testing of lower doses regardless of efficacy at higher doses. Unequal split of significance level. 12

13 Case study Structured family of hypotheses Four-armed trial comparing Three dose levels + standard of care Placebo + standard-of-care Three endpoints Primary endpoint: composite of CV death, MI or stroke Two key secondary endpoints Nine hypotheses Three doses (low, medium, high) Three endpoints per dose 13

14 Tailored multiple test procedure primary H 1 H 2 H 3 high dose medium dose low dose 14

15 Tailored multiple test procedure primary H 1 H 2 H 3 secondary H 4 H 5 H 6 high dose medium dose low dose 15

16 Tailored multiple test procedure primary H 1 H 2 H 3 secondary H 4 H 5 H 6 The nodes for secondary endpoints represent families of two null hypotheses related to the two key secondary endpoints high dose medium dose low dose 16

17 Tailored multiple test procedure primary H 1 H 2 H 3 secondary H 4 H 5 H high dose medium dose low dose 17

18 Tailored multiple test procedure 0.2α 0.4α 0.4α primary H 1 H 2 H 3 secondary H 4 H 5 H high dose medium dose low dose 18

19 Tailored multiple test procedure 0.2α 0.4α 0.4α primary H 1 H 2 H 3 secondary H 4 H 5 H high dose medium dose low dose 19

20 Tailored multiple test procedure 0.2α 0.4α 0.4α primary H 1 H 2 H 3 secondary H 4 H 5 H high dose medium dose low dose 20

21 Tailored multiple test procedure 0.25 primary 0.2α 0.4α 0.4α H 1 H 2 H secondary H 4 H 5 H high dose medium dose low dose 21

22 Tailored multiple test procedure 0.25 primary 0.2α 0.4α 0.4α H 1 H 2 H secondary H 4 H 5 H high dose medium dose low dose 22

23 Tailored multiple test procedure 0.25 primary 0.2α 0.4α 0.4α H 1 H 2 H secondary H 4 H 5 H high dose medium dose low dose Key secondary endpoints within each dose group will be tested with a weighted Bonferroni-Holm test at the available local significance level 23

24 Tailored multiple test procedure 0.25 primary 0.2α 0.4α 0.4α H 1 H 2 H secondary H 4 H 5 H high dose medium dose low dose 24 Key secondary endpoints within each dose group will be tested with a weighted Bonferroni-Holm test at the available local significance level Improvement of the multiple testing procedure by using weighted Dunnett s test for all intersection hypotheses that contain at least two of H 1, H 2 and H 3 for the same endpoint

25 Example rejection sequence 0.25 primary 0.2α 0.4α 0.4α H 1 H 2 H secondary H 4 H 5 H high dose medium dose low dose 25

26 Example rejection sequence primary 0.49α 0.45α 0.7 H 1 H 2 H secondary H 4 H 5 H α 0 0 high dose medium dose low dose 26

27 Example rejection sequence primary 0.55α 0.45α 0.7 H 2 H secondary 1 1 H 4 H 5 H high dose medium dose low dose 27

28 Example rejection sequence primary 0.835α H 2 H secondary 1 H 5 H α 0 high dose medium dose low dose 28

29 Case study Interim analyses: Bonferroni inequality for the repeated hypothesis testing Interim analyses at 50% and 75% information fraction I t A fixed Bonferroni split is used, with nominal overall significance levels α* t, t = 1, 2, 3, of 0.01% and 0.04% for the first and second efficacy interim analysis, leaving 2.45% for the final analysis. At each of the 3 analyses, the graphical testing procedure exploiting correlations between test statistics will be performed at the respective nominal level α* t, controlling the familywise type I error rate at the overall (one-sided) significance level α = 2.5%. H 1 H 2 α* t = 0.01% 0.04% 2.45% H 9 I t 0 50% 75% 100% Note: If a primary hypothesis is rejected, the respective secondary hypothesis cannot be tested at the full level α, even if the trial is stopped! 29

30 Summary and conclusions Confirmatory clinical trials are becoming increasingly more complex, often comparing multiple doses or treatments with a control for several primary and secondary endpoints. The multiple study objectives are reflected by structured families of hypotheses that are characterized by multiple groups of parent primary hypotheses and descendant secondary hypotheses. Novel graphical approaches for constructing and visualizing complex multiple test procedures with a focus on structured families of hypotheses are well suited to facilitate communication in clinical teams and to provide transparent decision strategies. Graphical procedures ensure strong control of the overall Type I error rate across all primary and secondary hypotheses. Multiple test procedure should be customized based on operating characteristics obtained via clinical scenario simulation. Ideally, EMA could provide in its new guidance a harmonized terminology and framework categorizing study objectives and endpoints with respect to their impact on approval and labeling and respective need for type 1 error control. 30

31 References Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009) A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine 28, Burman, C.-F., Sonesson, C. and Guilbaud, O. (2009) A recycling framework for the construction of Bonferroni-based multiple tests. Statistics in Medicine 28, Kordzakhia G. and Dmitrienko A. (2012) Superchain procedures in clinical trials with multiple objectives. Statist. Med. (early view) Maurer, W., Glimm, E., and Bretz, F. (2011) Multiple and repeated testing of primary, co-primary and secondary hypotheses. Statistics in Biopharmaceutical Research 3, Rohmeyer, K., Klinglmueller, F., Bornkamp, B. (2012) gmcp: Graph Based Multiple Test Procedures. R package version URL 31

32 32 Back-up

33 Case study Improvement of the multiple testing procedure by using weighted Dunnett s test for all intersection hypotheses that contain at least two of H 1, H 2 and H 3 for the same endpoint 0.2 α α 0.4 α Primary null hypothesis high Primary null hypothesis 0.03 dose 0.07 medium dose 0.13 dose Primary null hypothesis low ε ε ε First key secondary null hypotheses for high dose Second key secondary null hypotheses for high dose First key secondary null hypotheses for medium dose First Key secondary null hypotheses for low dose ε 1 1-ε 1 1-ε 1 Second key secondary null hypotheses for medium dose Second Key secondary null hypotheses for low dose

Disclaimer This presentation expresses my personal views on this topic and must not be interpreted as the regulatory views or the policy of the FDA

Disclaimer This presentation expresses my personal views on this topic and must not be interpreted as the regulatory views or the policy of the FDA On multiplicity problems related to multiple endpoints of controlled clinical trials Mohammad F. Huque, Ph.D. Div of Biometrics IV, Office of Biostatistics OTS, CDER/FDA JSM, Vancouver, August 2010 Disclaimer