Incorporating Virtual Patients into Clinical Studies

Transcription

1 Incorporating Virtual Patients into Clinical Studies Adam Himes, Tarek Haddad, Medtronic Laura Thompson 1, Telba Irony 2, Rajesh Nair 1 1 CDRH / FDA, 2 CBER / FDA on behalf of MDIC working group colleagues: Dawn Bardot, MDIC / Medtronic Anita Bestelmeyer, BD Dan Cooke, Boston Scientific Mark Horner, ANSYS Russ Klehn, St. Jude Medical Tina Morrison, OSEL / FDA Kyle Myers, OSEL / FDA Val Parvu, BD 1 MDIC.org

2 If it can be shown that these virtual patients are similar, in a precisely defined way, to real patients, future trials may be able to rely partially on virtual-patient information, thus lessening the burden of enrolling additional real patients. 2

3 3 Outline Motivation Virtual patients: physical + probabilistic modeling Combining virtual patients with clinical data Test drive: mock submission Review

4 4 COST Motivation Global demand: Demand for clinical evidence has never been higher Modeling advances: Our ability to simulate clinical outcomes has never been better Rising Demands: Duration Size Data 15% year/year New Factors: Diseases Markets Cost Models Kuntz, Insights on Global Healthcare Trends (2013) TIME J. Diabetes Sci Tech (2009) 3(1) Murbach, et.al BMES/FDA Frontiers Lee, et.al BMES/FDA Frontiers Dharia, et.al BMES/FDA Frontiers Yet it is still challenging to incorporate prior information into new studies

5 5 Virtual Patients as a New Source of Evidence Traditional Future Computer Human Virtual Patient Human Bench Animal Computer Animal Bench Integrate the virtual patient in clinical study design Use Bayesian statistical methods Build on 2010 FDA Guidance Maintain clinical and statistical rigor

6 6 What Makes a Virtual Patient? Like running a virtual clinical study! Physical Modeling Probabilistic Modeling Clinically Relevant Predictions Well Characterized Physics: Structural Electrical Heat transfer / fluid flow Knowledge of Biology / Physiology: Local device tissue interactions Failure modes Insulin response Variability: Age Gender Activity level Implant factors Physical tolerances Uncertainty: Sample size Measurement error/bias Model bias Clinically Relevant End Points: ICD lead fracture Orthopedic implant survival Coronary artery flow MRI heating Cardiac rhythm detection Blood glucose level

7 7 Sources of information for virtual patient models Virtual patient outcomes have to be exchangeable with something you d measure in a clinical study. Historical data Pilot studies, other geograhies, similar predicate products Real-world data Electronic medical records, claims data, observational studies Engineering and physiological models Credibility is the biggest hurdle.

8 8 Virtual Patient Example: ICD Lead Fracture Medical devices are particularly well suited to virtual patient modeling Local vs. systemic, often iterative, method of action is usually well understood Many applicable models, implantable defibrillator lead fracture is a good example Relevant, simple, public domain examples Haddad, et.al., Reliability Engineering and System Safety, 123 (2014): Swerdlow, et.al., JACC, 67 (2016):

9 9 Virtual Patient Example: ICD Lead Fracture INPUT OUTPUT in-vivo bending patient activity fatigue strength Field data Projection with 95% Confidence Interval Simulate many combinations of virtual patients & clinical trial Propagate variability and uncertainty to predict survival with confidence bounds Haddad, et.al., Reliability Engineering and System Safety, 123 (2014):

10 influence Bayesian Statistical Methods Provide a way to incorporate prior data into analysis of a clinical study Challenges: How much influence is given to the prior data? What if the clinical study data disagrees? study data prior data Solution: Method developed by MDIC working group Influence of prior data determined by agreement with study data Maintain statistical power with fewer patients discount function disagree agree (ideal state) disagree agree Haddad, et.al. (2017). J. Biopharm Stat, 27(6), DOI: /

11 Virtual patient weight Using a Discount Function to Control the Influence of Virtual Patients Influence determined by agreement between real patients and virtual patients less conservative Defined before starting the clinical study Maximum depends on model maturity Shape of function depends on desired characteristics more conservative If virtual and real patients disagree: Number of virtual patients decreases Eventually converts to a traditional study If virtual and real patients agree: disagree Statistical agreement agree Number of virtual patients increases up to prespecified maximum Haddad, et.al. (2017). J. Biopharm Stat, 27(6), DOI: /

12 12 Incorporate Virtual Patients: Step By Step current data 1. Compare virtual patient and current data p virtual patient data 2. Compute strength of prior using discount function α 0 This part is new p 3. Combine virtual patient and current data n total = n current + α 0 n VP combined data 4. Statistical analysis using combined data Haddad, et.al. (2017). J. Biopharm Stat, 27(6), DOI: /

13 13 Implementation: Mock Submission Collaboration between MDIC and FDA CDRH Division of Cardiovascular Devices Demonstrate the engineering and statistical framework for virtual patients MDIC sponsor team includes industry and FDA FDA review team, just like for a real device MDIC working group formed Mock submission team identified Mock submission informational meeting at FDA 2 nd Mock submission meeting (engineering model) at FDA 3 rd Mock submission meeting (clinical study) at FDA FDA MDIC project commitment initiated to develop methods 2017 for historical data

14 14 Mock submission details Hypothetical new ICD lead Changes to insulation thickness and conductor material, both affect fatigue life Expected fracture rate <1% at 18 months Single anatomical zone, single failure mode Clinical study design Objectives Fracture rate at 18 months < 3%, type I error < 10% Enrollment 200 initial patients, interim look every 30, maximum 400 patients. Up to 160 virtual patients (40%) Analysis No virtual patients, fixed amount, and with a discount function

15 15 Results Power = % chance of success that you deserve Type I error = % chance of success you don t deserve Traditional study with no virtual patients is underpowered 60% power / 3% type I error Fixed number of virtual patients has unacceptable type I error 96% power / 25% type I error Power suffers without virtual patients Type I error suffers without discount function Studies using virtual patients with a discount function have acceptable power AND type I error Function #1: 80% power / 5% type I error Function #2: 85% power / 10% type I error lower bound of virtual patient model performance performance requirement

16 16 What did we learn? MDIC working group formed Mock submission team identified Mock submission informational meeting at FDA 2 nd Mock submission meeting (engineering model) at FDA 3 rd Mock submission meeting (clinical study) at FDA FDA MDIC project commitment initiated to develop methods 2017 for historical data Three year process, required a very high level of collaboration Stakeholders have to engage in a different way Statistician, engineer, clinician, regulator all have a role in the study design The bandwidth of the regulatory communication process is a challenge engage early! Agreement on credibility of the prior is the most important topic Discount function allows for scaling according to prior credibility Introduces a different lens for evaluating historical data and engineering models All material available at

17 17 Model Credibility Level of trust in the model is driven by credibility. Credibility comes from verification and validation. ASME V&V40 risk-informed credibility assessment methodology Model influence: the contribution of the model to the decision relative to other available evidence Decision consequence: the significance of an incorrect decision (related to the device) Model Risk: combination of model influence and decision consequence for a context of use

18 18 Summary Virtual patients can improve the clinical decision process while exposing fewer patients to clinical trials Bayesian design with a discount function controls the influence of virtual patients The statistical methods are ready we just need the right applications! Model credibility is the most important thing