Third Plenary WHEN DO WE REALLY NEED RANDOMIZED CONTROLLED TRIALS? Sebastian Schneeweiss, MD, ScD Brigham & Women s Hospital and Harvard Medical School Boston, MA, USA When and how can Healthcare Database Analyses replace s in marketed medications? Sebastian Schneeweiss Division of Pharmacoepidemiology and Pharmacoeconomics, Dept. of Medicine, Brigham & Women s Hospital/ Harvard Medical School Disclosures: PI, Harvard Brigham & Women s Hospital Drug Safety Research Center (FDA) Co Chair, Methods Core of the FDA Sentinel System Consulting in past year: WHISCON LLC, Aetion Inc. (incl. equity) PI of research contracts to the Brigham & Women s Hosp.: Bayer, Genentech, Boehringer Ingelheim Grants/contracts from NIH, AHRQ, PCORI, FDA, IMI Advising FDA, EMA, PCORI, Health Canada 1
Real World Data in Pharmacoepidemiology Non experimental data 10% Research data Data collected PRIMARILY for research 90% Transactional data Data used SECONDARILY for research Example For purpose Data specifically for study purpose Framingham Study Cardiovas Health Study Slone Birth Defects Study Some registries Other purpose Data intended for other studies Nurses Health Study 1 Some registries Other purpose Clinical documentation EHR based studies NDI linkage Lab test databases Some registries Administrative Claims data studies Geocoding/census Database Studies CV Safety Example: BART 2
CV Safety Example: BART CV Safety Example: ENTRACTE Effectiveness Example: 3
Effectiveness Example: Effectiveness Example: HR = 0.66 (0.53 0.82) HR = 0.77 (0.54 1.09) 4
Effectiveness Example: Why did these database studies come to the HR = 0.66 (0.53 0.82) HR = 0.77 (0.54 1.09) same causal conclusion? Effectiveness Example: HR = 0.66 (0.53 0.82) HR = 0.77 (0.54 1.09) How confident are we that the study we plan will get it right? 5
Less helpful summarization of vs. non experim l: Hemkens et al. BMJ 2016 A corrected re analysis of Hemkens et al. BMJ 2016 Franklin JM, Rothman K, et al.: A Bias in the Evaluation of Bias Comparing Randomized Trials with Non experimental Studies. Epidemiology Methods 2017 Re analysis of treatment effects on mortality in RCD studies and s. For each clinical question, we present the relative odds ratio reported in trial evidence versus the corresponding RCD study. Effect estimates are presented when inverting treatment groups and ORs whenever the OR>1. 6
Following 3 guide posts 1. Active Comparator, same treatment modality 2. New Users 3. High-dimensional proxy adjustment result in 1. Clinical relevance 2. Meaningful causal statement 3. High validity 7
How to Investigatorcontrolled Datadependent 4. Outcome observable with specificity 5. Achieve covariate balance 6. Avoid known design flaws: a) Avoid immortal time bias b) Avoid adjusting for causal intermediates c) Avoid reverse causation d) Deal with time-varying hazards 7. Do robustness checks When to work with non experimental RWD studies for regulatory and payer decision making? Active comparator Same treatment modality comparator Outcome measureable How to Consider points 4 7 RWE software platforms improve quality, transparency Conduct feasibility analyses; Proceed if balance achieved in measured and UNMEASURED 8
When to work with non experimental RWD studies for regulatory and payer decision making? Active comparator Same treatment modality comparator Outcome measureable How to Consider points 4 7 RWE software platforms improve quality, transparency Conduct feasibility analyses; Proceed if balance achieved in measured and UNMEASURED There are >15 CV safety trials currently committed to FDA. If few of those could be replaced with RWE studies this would be a game changer. Effectiveness Example: HR = 0.66 (0.53 0.82) HR = 0.77 (0.54 1.09) What do we need to show to develop the confidence that the next RWE study will get it right? 9
Baseline Randomization Controlled measurement Seeming simplicity of design No randomization Non standardized observations Seeming complexity of design A priori confidence in causal conclusions What is our confidence level? What do we need to do to gain a priori confidence in causal conclusions? 10