The composite success

Transcription

1 The composite success Comparing drug development strategies with probabilities of success including benefit-risk assessment to inform decision-making Gaëlle Saint-Hilary 1 Véronique Robert 2, Mauro Gasparini 1 1 Politecnico Di Torino (Italy) 2 Institut de Recherches Internationales Servier (France) gaelle.sainthilary@polito.it PSI Conference, 15 May 2017 Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

2 Outline Introduction Predictive probability of success Composite definition of success Statistical significance Clinical relevance Positive benefit-risk balance Example in Major Depressive Disorders Conclusion Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

3 Introduction Decision-making problem Given what has been observed already, what are the chances of success of the drug development? We can calculate the Predictive Probability of Success (PPoS)... but what is a success? Usually defined as a statistically significant result (O'Hagan 2005, Gasparini 2013) Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

4 Introduction The success of a drug development The success of a drug development is driven by the conjunction between a valuable product and a successful development strategy A Marketing Authorization is usually conditioned by the success of the pivotal clinical trials, which must Reach statistical significance on their primary endpoint While showing a clinically meaningful effect of the drug The benefit-risk balance is a strong predictor of the long-term viability of a medicine, and a key element for the regulatory approval process Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

5 Introduction Composite definition of success We define the success of a drug development strategy as the simultaneous fulfillment of the following criteria in all pivotal trials: The statistical significance on the primary efficacy criterion A clinically meaningful effect on the primary efficacy criterion A positive benefit-risk balance versus the comparator(s) Given what has been observed already, what are the chances of success of the drug development? Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

6 Predictive Probability of Success (PPoS) Example: Success = statistical significance on the primary efficacy criterion δ=difference between treatments y m=observations in trial m d m=sample estimate of δ in trial m f =probability distribution Success of trial m: d m > c, c > 0 critical value PPoS = P(d m > c) = f (d m δ) f (δ y 1,..., y m 1 )dzdδ d m>c Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

7 Predictive Probability of Success (PPoS) Example for a Normal distribution After Trial 1, Predictions for Trial 2 Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

8 Predictive Probability of Success (PPoS) Example for a Normal distribution After Trial 1, Predictions for Trial 2 Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

9 Composite definition of success 1. Statistical significance on the primary endpoint Consider 2 treatments (i = 1, 2) assessed on the primary efficacy criterion, noted criterion 1 Criterion performance ξ i1 : performance of treatment i on criterion 1 Difference between treatments δ = ξ 11 ξ 21 Sample estimate in the considered trial d = sample estimate of δ Statistical significance The null hypothesis H 0 : δ 0 is rejected at level α if d > c, for c > 0 some critical value Clinical relevance The treatment effect is clinically meaningful if d > d T, for d T > 0 some clinical threshold Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

10 Composite definition of success 2. Clinical relevance on the primary endpoint Consider 2 treatments (i = 1, 2) assessed on the primary efficacy criterion, noted criterion 1 Criterion performance ξ i1 : performance of treatment i on criterion 1 Difference between treatments δ = ξ 11 ξ 21 Sample estimate in the considered trial d = sample estimate of δ Statistical significance The null hypothesis H 0 : δ 0 is rejected at level α if d > c, for c > 0 some critical value Clinical relevance The treatment effect is clinically meaningful if d > d T, for d T > 0 some clinical threshold Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

11 Composite definition of success 3. Positive benefit-risk balance Probabilistic Multi-Criteria Decision Analysis (pmcda) Waddingham 2016 ξ ij performance of treatment i on criterion j u j () used to normalize the performances on the criteria by mapping them on a 0 to 1 scale MCDA is identified by the EMA as the most comprehensive among the quantitative methodologies w j reflects the importance of the criteria u(ξ i, w) = w 1 u 1 (ξ i1 ) w n u n (ξ in ) Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

12 Composite definition of success 3. Positive benefit-risk balance Consider 2 treatments (i = 1, 2) assessed on n criteria (j = 1,..., n) of benefit and risk Benefit-risk utility score u(ξ i, w) = w 1 u 1 (ξ i1 ) w n u n (ξ in ) = n j=n w ju j (ξ ij ) Difference in benefit-risk utility scores u(ξ 1, ξ 2, w) = u(ξ 1, w) u(ξ 2, w) Sample estimate in the considered trial u(x 1, x 2, w) where x 1 and x 2 are sample estimates of ξ 1 and ξ 2 Positive benefit-risk balance Treatment 1 has a positive benefit-risk balance versus treatment 2 if u(x 1, x 2, w) > 0 Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

13 Predictive Probability of the Composite Success and of its components Predictive probabilities in one future trial comparing two treatments: Statistical significance on the primary efficacy criterion PPoS 1 = P(d > c) c > 0 critical value Clinically meaningful effect on the primary efficacy criterion PPoS 2 = P(d > d T ) d T > 0 clinical threshold Positive benefit-risk balance versus the comparator PPoS 3 = P( u(x 1, x 2, w) > 0) Predictive probability of composite success PPoS = P [(d > c) (d > d T ) ( u(x 1, x 2, w) > 0)] d, x 1, x 2 sample estimates of δ, ξ 1, ξ 2 in the future trial Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

14 Predictive Probability of the Composite Success and of its components Predictive probabilities in several future trials comparing two treatments: Statistical significance on the primary efficacy criterion PPoS 1 = ( ) S m=1 P [d m > c m δ] f (δ y)dδ c m critical value in trial m Clinically meaningful effect on the primary efficacy criterion PPoS 2 = ( ) S m=1 P [d m > dt m δ] f (δ y)dδ dt m clinical threshold in trial m Positive benefit-risk balance versus the comparator PPoS 3 = ( ) S m=1 P [ u(xm 1, x m 2, w) > 0 ξ 1, ξ 2 ] f (ξ 1 x 1 )f (ξ 2 x 2 )dξ 1 dξ 2 Predictive probability of composite success PPoS = ( ) S m=1 P [(d m > max(c m, dt m )) ( u(x m 1, x m 2, w) > 0) δ, ξ 1, ξ 2 ]. f (δ, ξ 1, ξ 2 y, x 1, x 2 )d (δ, ξ 1, ξ 2 ) d m, x m 1, x m 2 sample estimates of δ, ξ 1, ξ 2 in trial m Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

15 Example in Major Depressive Disorders Fictive case-study Effective treatment Dose-response relationship for efficacy and safety Hypokalemia may be a serious adverse effect Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

16 Example in Major Depressive Disorders Fictive case-study First strategy assessment: which dose has the best chances to succeed in Phase III? Predictive distributions Dose PPoS 1 PPoS 2 PPoS 3 PPoS (stat. signif.) (clin. relev.) (positive B/R) (composite) Low dose 74% 48% 88% 48% High dose 93% 78% 24% 24% Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

17 Example in Major Depressive Disorders Fictive case-study Strategy refinement: is it better to optimize the efficacy of the Low dose or to manage the risks for the High dose? Low dose with a possible dose-increase for non-responders High dose with potassium supplementation to prevent Hypokalemia Predictive distributions Regimen PPoS 1 PPoS 2 PPoS 3 PPoS (stat. signif.) (clin. relev.) (positive B/R) (composite) High dose suppl 93% 78% 95% 78% Dose increase 83% 59% 69% 58% Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

18 Concluding remarks The predictive probability of composite success and its components are helpful tools to compare development strategies and to inform decision-making in the pharmaceutical development Evidence-based approach Clinical data should be available: may not be appropriate in very early development Previous and future trials should be done in the same context (endpoint, duration, population) New hypotheses could be combined with the available evidence using priors What is a good PPoS? Phase/disease/project/team dependent Low amount of evidence PPoS close to 50% Uncertainty to take a decision Bayesian framework: PPoS are updated with the accumulation of knowledge from trial to trial Composite success criteria must be satisfied in each pivotal trial Consistent with the demand of replicability of the results But a similar approach could be applied at the development level (e.g. using meta-analyses) Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

19 Thank-you! Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

20 Main references I Gasparini M, Di Scala L, Bretz F, and Racine-Poon A. Predictive probability of success in clinical drug development. Epidemiology Biostatistics and Public Health, 10-1:e , Waddingham E, Mt-Isa S, Nixon R, and Ashby D. A Bayesian approach to probabilistic sensitivity analysis in structured benefit-risk assessment. Biometrical Journal, 58:28 42, OHagan A, Stevens JW, and Campbell MJ. Assurance in clinical trial design. Pharmaceutical Statistics, 4: , Saint-Hilary G, Cadour S, Robert V, and Gasparini M. A simple way to unify multicriteria decision analysis (MCDA) and stochastic multicriteria acceptability analysis (SMAA) using a dirichlet distribution in benefitrisk assessment. Biometrical Journal, 59(3): , Spiegelhalter DJ, Abrams KR, and Myles JP. Bayesian approaches to clinical trials and health-care evaluation. John Wiley & Sons Ltd, Chichester, UK, Chuang-Stein C, Kirby S, French J, Kowalski K, Marshall S, Smith MK, Bycott P, and Beltangady MA. Quantitative approach for making go/no-go decisions in drug development. Drug Information Journal, 45: , Stallard N, Whitehead J, and Cleall S. Decision-making in a phase II clinical trial: a new approach combining Bayesian and frequentist concepts. Pharmaceutical Statistics, 4: , Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

21 Main references II Zhang J and Zhang JJ. Joint probability of statistical success of multiple phase III trials. Pharmaceutical Statistics, 12: , Wang M, Liu GF, and Schindler J. Evaluation of program success for programs with multiple trials in binary outcomes. Pharmaceutical Statistics, 14: , Hong S and Shi L. Predictive power to assist phase 3 go/no go decision based on phase 2 data on a different endpoint. Statistics in Medicine, 31: , Ashby D and Smith A. Evidence-based medicine as Bayesian decision-making. Statistics in Medicine, 19: , EMA. Benefit-risk methodology project. work package 4 report: Benefit-risk tools and processes IMI PROTECT. IMI PROTECT Work package 5: Benefit-risk integration and representation Mt-Isa S, Ouwens M, Robert V, Gebel M, Schacht A, and Hirsch I. Structured benefit-risk assessment: a review of key publications and initiatives on frameworks and methodologies. Pharmaceutical Statististics, 15: , Antonijevic Z. Optimization of Pharmaceutical R & D Programs and Portfolios. Springer International Publishing, Switzerland, Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

22 Back-up slides Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

23 Example in Major Depressive Disorders Fictive case-study Distributions of the parameters Parameter Sample estimate Prior Likelihood Posterior HAM-D 17 mean total score in each arm (i = 1,..., 3) ( ξ i1 m i N(0, σ0 2) N(ξ i1, σi 2) N σ0 2 m i, HAM-D 17 mean total score, difference versus placebo (i = 1, 2 ) δ i = ξ 31 ξ i1 d i = m 3 m i N(0, s 2 0 ) N(δ i, s 2 i ) N ( σ 2 0 +σ2 i σ 2 0 σ2 i σ 2 0 +σ2 i Occurrence of adverse events in each arm (i = 1,..., 3 ; j = 2,..., 6) ξ ij r ij /n ij Beta(1, 1) Bin(n ij, ξ ij )/n ij Beta(r ij + 1, n ij r ij + 1) σ 2 0 = 104 ; s 2 0 = 2 σ2 0 = s 2 0 s 2 0 +s2 i d i, s 2 0 s2 i s 2 0 +s2 i ) ) Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24

24 Example in Major Depressive Disorders Fictive case-study Strategy refinement The two new strategies and their statistical assumptions are as follows: (i) High dose with potassium supplements. The predictions are based on the posterior distribution of ξ 32 obtained for the placebo for Hypokalemia, and on the posterior distributions obtained for the High dose for all the other criteria. (ii) Dose increase. According to the clinicians, 30% to 40% of the patients would increase to the High dose in the Phase III study, therefore a new parameter with a uniform distribution ζ U[0.3, 0.4] is introduced in the model as the proportion of patients receiving the High dose. The distributions of the parameters associated to the efficacy and safety criteria are obtained from the distributions of the initial parameters: (1 ζ) ξ 1j + ζ ξ 2j for j = 1,..., 6. Gaëlle Saint-Hilary (PoliTo & Servier) Decision-making - Composite success 15 May / 24