Utilities and Pitfalls of Modeling & Simulation in Pivotal Trials

Transcription

1 Utilities and Pitfalls of Modeling & Simulation in Pivotal Trials H.M. James Hung, Ph.D Div. of Biometrics I, OB/OTS/CDER U.S. Food and Drug Administration Presented in PhRMA/FDA Workshop, October 28, 2009, Washington, DC Disclaimer The views and opinions expressed in the following PowerPoint slides are those of the individual id presenter and should not be attributed t to Drug Information Association, Inc. ( DIA ), its directors, officers, employees, volunteers, members, chapters, councils, Special Interest Area Communities or affiliates, or any organization with which the presenter is employed or affiliated. These PowerPoint slides are the intellectual property of the individual presenter and are protected under the copyright laws of the United States of America and other countries. Used by permission. All rights reserved. Drug Information Association, DIA and DIA logo are registered trademarks or trademarks of Drug Information Association Inc. All other trademarks are the property of their respective owners. 1

2 Acknowledgment Norman Stockbridge John Lawrence Steve Bai Jialu Zhang Cherry Liu Fanhui Kong Valeria Fredlin Disclaimer The views expressed in this presentation are not necessarily of US Food and Drug Administration J. Hung, 2009 PhRMA/FDA Wkshop 3 Outline Drug development process Utilities of M & S Pitfalls of M & S Model diagnostics Conclusions J. Hung, 2009 PhRMA/FDA Wkshop 4 2

3 Drug Development Process Exploratory/Learning phase Confirmatory (pivotal) phase Big gap in between J. Hung, 2009 PhRMA/FDA Wkshop 5 Exploratory/Learning Phase M & S using biological or pharmacological model on markers or parameters is vital to learn possible pathways by which the test product may induce benefit or harm ~~~~ Clinical i l hypotheses generated ~~~~ J. Hung, 2009 PhRMA/FDA Wkshop 6 3

4 Confirmatory Phase In pivotal trials, the clinical hypotheses are tested to give a yes/no answer Seldom there is a model for clinical endpoint Statistical test: least model assumption - scientific validity robust to assumption M & S often conducted with care J. Hung, 2009 PhRMA/FDA Wkshop 7 Drug Development Process From learning to confirming, there is a big gap that is often impossible to bridge, i.e., between markers and clinical endpoints, surrogacy problem Projection from markers data to clinical outcomes often does not realize J. Hung, 2009 PhRMA/FDA Wkshop 8 4

5 Utilities of M & S Select best statistical testing procedure - stable type I error rate, robust statistical power Study relationship between outcome and covariates for description, e.g., - treatment effect over time, dropouts - heterogeneity of treatment effect.. J. Hung, 2009 PhRMA/FDA Wkshop 9 Utilities of M & S Select best statistical testing procedure - stable type I error rate, robust statistical power Study relationship between outcome and covariates for description, e.g., - treatment effect over time, dropouts - heterogeneity of treatment effect.. J. Hung, 2009 PhRMA/FDA Wkshop 10 5

6 Case 1 J. Hung, 2009 PhRMA/FDA Wkshop 11 Mean effect: placebo subtracted effect SD = 10 J. Hung, 2009 PhRMA/FDA Wkshop 12 6

7 J. Hung, 2009 PhRMA/FDA Wkshop 13 Utilities of M & S Select best statistical testing procedure - stable type I error rate, robust statistical power Study relationship between outcome and covariates for description, e.g., - treatment effect over time, dropouts - heterogeneity of treatment effect.. J. Hung, 2009 PhRMA/FDA Wkshop 14 7

8 Case 2 Avalide label J. Hung, 2009 PhRMA/FDA Wkshop 15 Avalide label J. Hung, 2009 PhRMA/FDA Wkshop 16 8

9 Graphs based on proper empirical modeling may help to convey useful information: - probability of achieving BP goal depends on baseline BP level - probability of achieving BP goal remains low, even with taking a combination of two drugs Q: What are required for proper empirical modeling? Model Diagnostics J. Hung, 2009 PhRMA/FDA Wkshop 17 Pitfalls of M & S Modeling can lead to misleading interpretation of trial results M & S can lead to shaky trial design and ultimately fail the trial Inadequate M & S can lead to misleading description of trial results J. Hung, 2009 PhRMA/FDA Wkshop 18 9

10 Pitfalls of M & S Modeling can lead to misleading interpretation of trial results M & S can lead to shaky trial design and ultimately fail the trial Inadequate M & S can lead to misleading description of trial results J. Hung, 2009 PhRMA/FDA Wkshop 19 Case 3 In an NDA, 1 st trial findings: Primary endpoint failed (p = 0.60). A secondary endpoint seemed interesting - Response over 56 days looks linear - Difference between drug and placebo in rate of change over time (i.e., slope) p = J. Hung, 2009 PhRMA/FDA Wkshop 20 10

11 2 nd trial (to replicate the 1 st trial finding): Rate of change (assume linear), p <.001 But, response profile is quadratic - quadratic curve show a significant ifi curvature - rate of change meaningless b/c it varies over time - further exploration suggests response may be linear over first 21 days and drug effect, if any, is seen for first 7 days J. Hung, 2009 PhRMA/FDA Wkshop 21 3 rd trial (to replicate the 2 nd trial finding): Rate of change: p = , depending on how to handle dropouts Bt But, reviewer analysis: - challenge linearity - other sensitivity analyses showed p = to pre-specified analysis p = 0.12 J. Hung, 2009 PhRMA/FDA Wkshop 22 11

12 For completers and two dropouts with complete data Placebo Drug J. Hung, 2009 PhRMA/FDA Wkshop 23 Responses of dropouts J. Hung, 2009 PhRMA/FDA Wkshop 24 12

13 Model Diagnostics Model checking and diagnostics is critical to utility of M & S method, particularly for pivotal (or confirmatory) trials What kind of diagnostics are necessary: - Good of fit test, lack of fit test,. - Model residuals assessment - Influence assessment J. Hung, 2009 PhRMA/FDA Wkshop 25 Model diagnostics for Case 2 J. Hung, 2009 PhRMA/FDA Wkshop 26 13

14 J. Hung, 2009 PhRMA/FDA Wkshop 27 J. Hung, 2009 PhRMA/FDA Wkshop 28 14

15 J. Hung, 2009 PhRMA/FDA Wkshop 29 J. Hung, 2009 PhRMA/FDA Wkshop 30 15

16 Conclusions Model & Simulation for pivotal trial is usually done to evaluate the properties of trial design and analysis strategy For testing clinical hypotheses, statistical tests should avoid model assumptions as much as possible, be robust to model assumptions J. Hung, 2009 PhRMA/FDA Wkshop 31 For descriptive purposes, modeling requires thorough model diagnostics according to the well-founded statistical principles i Caveat: use of the same data to develop a model and then test the hypothesis or for prediction may substantially inflate type I error rate - separate data into test and training sets J. Hung, 2009 PhRMA/FDA Wkshop 32 16