Including Real World Evidence (RWE) in network meta-analysis

Size: px

Start display at page:

Download "Including Real World Evidence (RWE) in network meta-analysis"

Adrian Johnson
6 years ago
Views:

1 Including Real World Evidence (RWE) in network meta-analysis David Jenkins, Reynaldo Martina, Sylwia Bujkiewicz, Pascale Dequen & Keith Abrams Department of Health Sciences,

2 Background GetReal is a three-year project of the Innovative Medicines Initiative (IMI), a EU public-private consortium consisting of pharmaceutical companies, academia, regulators (e.g. EMA), HTA agencies (e.g. NICE, HAS, EMA and ZIN) & patient organisations GetReal aims to show how RWE (not just RCTs) generation, collection and synthesis could be used earlier in pharmaceutical R&D and the healthcare decision making process to increase efficiency A case study in multiple sclerosis (MS), a neurological disorder of the central nervous system. Disease course is characterised by recurrent neurological attacks, known as relapses (Olek, 202)

3 Augmenting RCTs with RWE (Relative) Clinical Effectiveness RCTs RCT (A v P) RCT2 (B v P) RCT3 (C v P) Real World Evidence Registry/Obs (A v B v C v P) Adjusting for for Adjusting casecasemix/confoundin mix/confoundin g (if (if IPD IPD are are g available) available)

4 Including RWE in NMA Systematic review & NMA undertaken up to & including fingolimod (Gilenya) HTA submissions in conjunction with Novartis Outcome of interest is annualised relapse rate Inclusion of RWE as well as RCTs (naïve pooling) Assessment of inconsistency and heterogeneity Regression analysis Assessment of different methods for weighting RWE due to potential biases; Power transform prior approach (sensitivity analysis) Hierarchical model (including bias adjustment as above)

5 RCT network Fingolimod(.25) 3 Fingolimod(0.5) Avonex 2 2 Natalizumab Placebo Rebif22 2 Betaferon Rebif44 2 GA

6 RWE network Rebif44 Natalizumab Rebif Placebo Avonex 7 4 GA Betaferon

7 RCT + RWE network diagram Fingolimod(0.5) Fingolimod(.25) 3/0 2/0 /0 /0 Natalizumab 2/0 Avonex 0/ 0/ /0 0/ / 0/4 /7 /2 /0 0/4 Rebif22 / Rebif44 0/ Placebo 0/ /0 2/2 / 0/4 0/2 0/2 2/4 0/2 Betaferon GA

NMA RCT, RWE & Combined Assuming a Poisson distribution, the basic model is, is the number of relapses in arm k of study i refers to the number of patient years and therefore, is the annualised

8 NMA RCT, RWE & Combined Assuming a Poisson distribution, the basic model is, is the number of relapses in arm k of study i refers to the number of patient years and therefore, is the annualised relapse rate (ARR) of arm k in study i Hence, is the log relative ARR of the treatment in arm k of study i and is the baseline intervention for study i is the additional part of the model added for regression, with the covariate X for each arm k of study i

9 NMA RCT, RWE & Combined

Issue about whether to use a single, independent or exchangeable regression term(s) for each treatment

10 NMA RCT and RWE combined: Regression Evidence to suggest a difference between RCTs and RWE Around.43 more relapses are observed in the RCTs This method looks more at the importance of the covariate Issue about whether to use a single, independent or exchangeable regression term(s) for each treatment in order to reflect what the true effect is in a real world population Increase/decrease in uncertainty of treatment

11 Allowing for potential biases in RWE Power transform prior () Following Ibrahim & Chen, Stat. Sci ():46-60 Power Transform prior approach RWE is weighted by term 0 is degree of weighting = 0 total discounting = accept at face value Evaluate for a range of values of Typically raise the power of the likelihood for the RWE

Power transform prior (2) Consider the annualised relapse rate ratio (ARRR) and assuming, then the overall joint posterior distribution is given by, RCT likelihood

12 Power transform prior (2) Consider the annualised relapse rate ratio (ARRR) and assuming, then the overall joint posterior distribution is given by, RCT likelihood stays the same, however, the RWE likelihood changes Assuming a Poisson distribution, the RWE loglikelihood (LL) is then, where h indexes the different values of alpha

13 Power transform prior - results

14 Power transform prior results (2) Uncertainty increases the more weight given to the RWE Two treatments have smallest CI at alpha=0.2

15 Power transform prior (3) Power prior model only adjusts the weighting of the RWE by the same amount for each study Can be altered to give different weightings to different studies Key issue with this method is that of which alpha to use. A valid reason would be needed Should not just pick the alpha with smallest credible interval scoring each study on quality linking alpha to empirical evidence regarding bias (Turner et al, 2009) Possibly more for illustrative purposes or to back up/support evidence and justify other approaches or conclusions

Hierarchical NMA () An additional level to account for study type can be added (Prevost et al, 200; Owen et al, 205) Allows for adjustments to systematic bias and

16 Hierarchical NMA () An additional level to account for study type can be added (Prevost et al, 200; Owen et al, 205) Allows for adjustments to systematic bias and weighting by various design Allows for the both within and between study type heterogeneity Allows for the imprecision in the estimated betweenstudy/type variances

Hierarchical NMA (2) Let and represent the treatment effect of drug k against placebo based on the RCT evidence and RWE, respectively, then, where is the mean treatment effect of drug k and τ is the

17 Hierarchical NMA (2) Let and represent the treatment effect of drug k against placebo based on the RCT evidence and RWE, respectively, then, where is the mean treatment effect of drug k and τ is the precision representing between study heterogeneity Exchangeability can now be relaxed between RCT and RWE by adjusting for bias in study designs An additive factor can be used in order to adjust for overestimation Here α inflates the variance to account for unknown bias Both α and β can be based on empirical evidence from systematic differences

Hierarchical NMA - Results Each treatment rate ratio compared to placebo for naïve pooling, hierarchical and hierarchical with power prior CIs increase to

18 Hierarchical NMA - Results Each treatment rate ratio compared to placebo for naïve pooling, hierarchical and hierarchical with power prior CIs increase to account for between-study type heterogeneity Uncertainty is more consistent across all alpha values for the hierarchical model with power prior (here alpha=0.9)

19 Hierarchical NMA (3) Care needs to be taken as to not confuse within and between study relationships The RCT and RWE network could be disconnected Captures more of the uncertainty (not overestimating) Extend methods to incorporate either study/individual-level characteristics The hierarchical model can be further extended, for example, types of RWE, i.e. Phase IV, cohort, etc

20 Summary Power prior More for sensitivity and exploration to understand more about what is happening when RWE is includes Hierarchical Advised as a better option to regression because it accounts for heterogeneity between and within study type (and therefore does not over-estimate treatment effects) Add the power prior to this method for a clearer understanding Future work Evaluating and applying other methods to include RWE (eg multivariate NMA) Combining long term RWE with standard (shorter) RCTs to help determine longer term outcomes

21 References Olek, M. J. (202). Epidemiology and clinical features of multiple sclerosis in adults. UpToDate. Waltham: UpToDate Owen, R. K., Tincello, D. G., & Keith, R. A. (205). Network Meta-Analysis: Development of a Three-Level Hierarchical Modeling Approach Incorporating Dose-Related Constraints. Value in Health, 8(), Prevost, T. C., Abrams, K. R., & Jones, D. R. (2000). Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening. Statistics in medicine, 9(24), Turner, R. M., Spiegelhalter, D. J., Smith, G., & Thompson, S. G. (2009). Bias modelling in evidence synthesis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 72(), 2-47 World Health Organization. Atlas multiple sclerosis

GetReal: Clinical effectiveness in drug development

GetReal: Clinical effectiveness in drug development Mike Chambers Matthias Egger WP4 Leaders GSK, UK Univ. of Berne, Switzerland BBS/EFSPI Seminar Basel 04Jun13 Egger/Chambers: real world evidence in HTA