By: Harrison Reeder and Kamrine Poels Mentor: Dr. Kathryn Chaloner

Transcription

1 Evaluating an Adaptive Clinical Trial with Quantitative Endpints, Sample Size Re-estimatin, Sequential Mnitring fr Efficacy, and Mnitring fr Futility By: Harrisn Reeder and Kamrine Pels Mentr: Dr. Kathryn Chalner

2 Outline What exactly des that title mean? Basic Clinical Trial design Interim Mnitring fr Efficacy 3 schemes fr interim mnitring fr efficacy Interim mnitring fr futility Adaptive sample size re-estimatin Simulatin Study f Design Perfrmance Cnclusin

3 Clinical Trial Design: The Basic Case The mst basic element f clinical trial design is determining an adequate sample size Calculating sample size requires specifying: apprximate variance f utcmes the desired Type I errr rate minimum clinically meaningful treatment effect desired pwer t detect that effect pwer.t.test()

4 Interim Mnitring fr Efficacy Why use interim mnitring? Cmplicatins f interim mnitring Interim mnitring inflates Type I errr Slutin: Change bundary f significance Taken frm Intrductin t Randmized Cntrlled Clinical Trials by Jhn Matthews

5 Schemes fr Interim Efficacy Mnitring Pcck "cnstant" bundaries sets cnstant p-value bundary t use at every mnitring pint Earlier rejectin is easier, but final test is stringent O'Brien-Fleming bundaries makes rejectin harder at earlier pints and easier as trial prgresses Fleming-Harringtn-O'Brien bundaries middle-grund between abve strategies Bundary First Interim Secnd Interim Third Interim Final pint Pcck O-F F-H-O

6 Interim Mnitring fr Futility Why mnitr fr futility? Cnditinal pwer Estimates prbability f having significant results given bserved data and (design) assumptins If prbability is lwer than a specified threshld, then trial is stpped

7 Adaptive Sample Size Recalculatin Early estimate f respnse variance is difficult T accunt fr difference between estimate and true value, this design uses bserved estimated variance halfway int trial t re-estimate sample size Investigatrs can set a maximum sample size fr each grup

8 Research Questin: Hw des ur design perfrm? Using simulatin, we cmpare the design t designs withut the features described We als cmpare the merits f the three interim mnitring schemes Values f interest: Bias f final treatment effect estimate True cnfidence f nminal 95% Cnfidence Interval True Type I errr True pwer Distributin f stpping pints

9 Designing the Simulatin First Interim Secnd Interim Third Interim Sample Size is 9 Check fr efficacy Sample Size is 18 Check fr efficacy Check fr futility Final sample size is recalculated Sample size is Final+18 if recalculated 2 Withut sample size recalculatin, size is 28 Check fr efficacy Mtivating Study: Effect f Sleeping Drug in Adlescents and Yung Adults with Autism Spectrum Disrder Design assumptins: Mean treatment effect: 32 minutes Respnse standard deviatin: 36 minutes Final Pint Sample size is Final 50 if recalculated Withut sample size recalculatin, size is 35 Check fr efficacy Simulatin seed: 42 Cnditinal pwer seed: 123

10 Effect f Interim Mnitring fr Efficacy (Withut Sample Size Re-estimatin r Futility Mnitring) Ending sample size < 35 per grup because we can stp at earlier interim pints when results are significant Bias f estimated treatment effect is psitive (verestimates by ~10% n average)

11 Effects f Interim Mnitring fr Futility (Withut Sample Size Re-estimatin) Large drp in true Type I errr frm ~0.05 t ~0.01 (mre pprtunities t stp an ineffective trial frm fllwing thrugh t the end and having significance by chance) Effects mediated by cnditinal pwer bund Smaller stpping pint sample size when respnse variance is larger than expected Chance f stpping early fr futility, even if alternative is true, explains a slight drp in true pwer

12

13

14 Effects f Sample Size Recalculatin If the design variance is greater than r equal t the true variance, recalculatin tends t decrease the ending sample size Likewise, underestimated variance leads t a larger required sample size Pwer fllws a similar trend

15

16

17 Cmparisn f Bundary Types Pcck Highest Type I errr Highest bias Lwest pwer Smallest sample size (i.e., best chance f finding efficacy early) O'Brien-Fleming and Fleming-Harringtn-O'Brien Similar results acrss measures and assumptins O'Brien-Fleming bundary is mre cmmnly used

18 Overall Evaluatin f Our Design These characteristics shw the design's ptential value in Phase II trials: Minimizes Type I errr rate Maintains pwer when variance estimate is t lw May decrease sample size required t reach a cnclusin Limitatins: Sample size re-estimatin ptentially increases cst Gives biased estimate f treatment effect

19 Hw Des Our Design Cmpare t Interim Mnitring fr Efficacy Alne? If assumptins are accurate, with ur design: Median ending sample size is smaller Pwer is slightly lwer, but cmparable Type I errr rate is lwer (imprtant fr Phase 2 trials) If assumptins are inaccurate (verestimated effect size and underestimated variance): Ending sample size tends t be larger (mre expensive) Pwer is higher (thugh verall bth are much lwer) Type I errr rate is lwer

20

21 O'Brien-Fleming Graphs

22 Cnclusin: "Is ur design better fr the mtivating study?" Yes! Minimizing Type I errrs is imprtant in Phase II trials, which is achieved in ur design Treatment effect and respnse variance are nt easily estimated in the mtivating study Our design's ability t maintain pwer and keep errr rates lw even with inaccurate design assumptins is beneficial Limitatin: Ptential fr higher re-estimated sample size may increase cst f trial