Protocol. This trial protocol has been provided by the authors to give readers additional information about their work.

Transcription

1 Protocol This trial protocol has been provided by the authors to give readers additional information about their work. Protocol for: Resnic FS, Majithia A, Marinac Dabic D, et al. Registry-based prospective, active surveillance of medical-device safety. N Engl J Med 2017;376: DOI: /NEJMoa

2 Supplementary Material for: Registry-Based Prospective Active Medical Device Safety Surveillance This supplement contains the following items: 1. Original study protocol and statistical analysis plan Approved 5/25/ Signal Persistence Protocol Amendment - Approved 12/4/ Summary of all protocol and statistical analysis changes

3 Evaluation of Automated Active Surveillance of Cardiovascular Devices: American College of Cardiology National Cardiovascular Data Registry (ACC-NCDR) Protocol Version: 1.0 Dated May 25, 2012 Principle Investigator: Frederic S. Resnic, MD MSc Coordinating Site: Brigham and Women s Hospital, Boston MA Harvard Medical School, Boston MA Supported by: FDA/HHS F C 2

4 Table of Contents Introduction... 4 Objectives... 4 System Design and Prior Experience... 4 Data Security and System Configuration... 5 Scientific Steering Committee... 5 Proposed Safety Analyses in Phase I... 5 Safety of the Mynx vascular closure device (AccessClosure Inc., Mountain View, CA) Comparative safety and effectiveness of mechanical thrombectomy catheters (including Export (Medtronic), Pronto (Vascular Solutions) and others)... 6 Statistical Methodology... 6 Final Device Safety Reports... 7 Human Subjects Institutional Review... 7 Project Funding:... 7 Appendices... 8 Appendix A - Operations Manual... 8 References

5 Introduction The Brigham and Women s Hospital research program in post-market surveillance for medical device safety has focused on strategies that utilize medical informatics to build an automated safety surveillance infrastructure which support monitoring of the safety of new devices and drugs after initial market release. With the rapid growth of electronic medical records in the U.S. and the rest of the world and the additional availability of clinical outcomes data repositories large and deep clinical datasets now provide the opportunity to enable computerized tracking of exposure to new devices and medications, and to understand the risk-adjusted adverse event rates for those new products. This new form of automated active surveillance is one way of addressing an unmet public health need in the U.S. and elsewhere. The Data Extraction and Longitudinal Trend Analysis (DELTA) system is a collection of integrated computer applications linked to a robust Microsoft SQL database and is used to register patient characteristics, exposures to medical devices and medications, and the outcomes to treatments through read-only access to the underlying data-source (such as a clinical registry or electronic medical record system). DELTA incorporates a variety of conventional frequentist and Bayesian methods to perform risk-adjusted prospective surveillance analyses, including survival studies and extensive propensity matching algorithms 7. DELTA can support a number of prospective safety analyses simultaneously and incorporates sophisticated security and de-identification algorithms to protect sensitive health details and uphold legislation regarding data ownership. When taken collectively, the system s attributes and features render it an innovative, viable and powerful tool toward monitoring the safety of new drugs and therapies. Since the project s inception, the DELTA system has been utilized in assessing the safety of recently introduced cardiovascular devices in a statewide high quality clinical outcomes database 8. Additional ongoing projects include a Veteran s Administration CART-CL dataset analyses of implantable cardiovascular devices, an FDA sponsored pilot project exploring the durability and safety of prosthetic hip implants, and several projects with University of California San Diego focusing on comparative effectiveness research of cardiovascular medications in common conditions such as atrial fibrillation and venous thrombo-embolic disease. Objectives The objective of this study is to implement a DELTA prospective safety surveillance system at the American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR) that integrates with the CathPCI clinical outcomes registry and is capable of prospectively monitoring the safety of selected cardiovascular devices captured by the registry. The study will focus on the evaluation and safety of two types of interventional cardiology devices used during percutaneous coronary intervention (PCI) procedures as a proof of concept exercise. System Design and Prior Experience The foundation of the integrated data-sharing network will be the Data Extraction and Longitudinal Trend Analysis (DELTA) system developed at Brigham and Women s Hospital The DELTA system 4

6 has evolved into a networkable medical product automated prospective safety surveillance system supporting a wide range of frequentist and Bayesian statistical inference methods 7. The system has been validated in multiple domains including medication safety surveillance medication interaction surveillance and multiple prospective medical device safety surveillance environments 123. The DELTA Network study of interventional cardiology devices was launched in March 2010 among four medical centers Massachusetts, demonstrating the capability of a distributed prospective surveillance system in the monitoring of post-market adverse events following interventional cardiology device implantation. Data Security and System Configuration The DELTA system can be implemented completely within the ACC-NCDR network firewall, with readonly access of the CathPCI dataset. The DELTA system will be accessible only by authorized users working within the NCDR intranet or authorized users accessing the network securely through VPN access. DELTA configuration will be supported by the data analysis center at Brigham and Women s Hospital, but no access to the underlying data or summary level information need be provided outside of the NCDR intranet (See Appendix A) Scientific Steering Committee At the initiation of the project, a scientific steering committee will be established with the comprised of three representatives from NCDR, two representatives from the data coordinating center (Drs. Resnic, Matheny), and two representatives from FDA CDRH, the sponsor of the study. The scientific steering committee will be responsible for the approval of proposed safety studies, final acceptance of study protocols, and will have oversight of the overall performance and execution of the study as well as review and approval of any publications resulting from the project. Proposed Safety Analyses in Phase I Phase I of the NCDR-DELTA project will explore the safety of two types of devices used during interventional cardiology procedures, which have not previously been analyzed using comparative safety techniques. The analyses will be approved by the study steering committee, established at the initiation of the project. Both proposed analyses will be performed using CathPCI version 4 dataset definitions, to maximize consistency of outcome and clinical covariate definitions. The proposed studies will span from the initiation of CathPCI v4 through two quarterly submissions following the implementation of the DELTA system, to permit testing of the prospective and continuous surveillance capabilities of the system. Initial proposed safety analyses: Safety of the Mynx vascular closure device (AccessClosure Inc., Mountain View, CA). The device was approved by FDA for market release in May 2007 and has become a widely used technique for managing femoral vascular access sites in patients undergoing PCI. The device design permits completely extravascular hemostastis, which permits the device to be used in a wide variety of patients 5

7 following PCI. However, preliminary data from the Massachusetts PCI registry indicates a possible increased risk of local vascular complications with the Mynx device relative to other vascular closure devices (VCD) using prospective concurrent propensity matched control analyses. 910 We propose to study the rates of access sites bleeding (8055) and any local vascular complications (8035, 8060, 8070), as well as the rates of transfusion (840), over time, for the Mynx device relative to other VCD following PCI procedures. In addition, pre-specified secondary analyses will explore the institutional and operator specific learning curves for the Mynx device as reflected by changes in overall vascular complications and bleeding as defined above. Comparative safety and effectiveness of mechanical thrombectomy catheters (including Export (Medtronic), Pronto (Vascular Solutions) and others). While randomized clinical trials and metaanalyses have demonstrated improved survival and other clinical outcomes when mechanical thrombectomy is performed as part of primary PCI for STEMI 11, differences in performance of the various mechanical thrombectomy catheters have not been reported. Given the size of the ACC-NCDR CathPCI dataset, a DELTA analysis focused on the comparative effectiveness and safety of the alternative mechanical thrombectomy catheters is feasible. We propose to study the rates of major adverse cardiac events, including death, emergent bypass surgery, and emergent repeat PCI procedures for patients using the various mechanical thrombectomy devices, through a propensity matched concurrent control methodology. In addition, we propose to explore institutional and operator specific learning curves, as reflected by rates of major adverse cardiac events with accumulating experience with specific devices. Statistical Methodology The primary analyses will be based on a propensity matched analysis in which the patients exposed to the device of interest will be compared to similar patients receiving an alternative comparator device within 6 months of the case s exposure. Proposed alert thresholds for (simulated prospective) automated surveillance will include any significant increase (p-value < 0.05 after correction for multiple comparisons) in the adverse event rates through time of hospital discharge following exposure implant. The data analysis will be performed within the DELTA application, and will be available for review to authorized users of the system. The DELTA system will monitor each of the specified devices for the adverse events noted above using retrospective data from 9 calendar quarters prior to study initiation through two calendar quarters of prospectively collected CathPCI data. De-identified CathPCI ( limited ) datasets will be uploaded to DELTA at three times. The total study period will therefore include 11 calendar quarters (33 months) of CathPCI data. The initial data upload will include data 9 calendar quarters retrospective data, and will be analyzed in a simulated prospective manner, with a cumulative analysis repeated at each calendar quarter. Two additional data uploads, each including one additional calendar quarter s worth of data, will be made available as soon as the CathPCI limited dataset for these quarters is finalized by NCDR. The cumulative analyses will be repeated prospectively for each of these additional data uploads. Each analysis will include pre-specified sensitivity analyses that will be triggered whenever an alert is signaled by DELTA indicating a higher than expected adverse event rate for a device-outcome analysis. Sensitivity analyses will include alternative event rate estimation (including multivariate 6

8 regression), periodic analyses, and exploration of predefined patient cohorts (including: patients>70yrs, diabetics, women). Final Device Safety Reports The analytic coordinating center at Brigham and Women s Hospital will prepare two interim and one final device safety summary reports for review by the scientific steering committee. It is anticipated that the final report will serve as the basis for two manuscripts (one addressing the comparative safety of each device class tested) which will be prepared within 6 months of the completion of the analysis and submitted for publication in peer reviewed journal. Members of the scientific steering committee will be invited to participate in the writing and review of each of the proposed manuscripts. Human Subjects Institutional Review The study protocol, and all other study related materials will be submitted for IRB review and approval per the institutional requirements of NCDR, FDA and Brigham and Women s Hospital. Project Funding: Funding for the hardware, DELTA software and analytic oversight by Brigham and Women s Hospital staff is supported, in part, by FDA contract to Brigham and Women s Hospital, Boston Massachusetts (# HHSF C Mod 0004). 7

9 Appendices Appendix A - Operations Manual I. Regulatory Documents All regulatory documents pertaining to this research project for each site will be kept in a designated regulatory binder, in a secure location per site IRB requirements. Included in the Investigators Binder at the coordinating site (BWH/HMS) will be copies of all IRB application and approval letters from the participating sites as well as yearly continuing review documentation and approval letters. II. Data Use/Access 1. All access to site level data will be restricted to site-specific authorized users of the DELTA local system(s). There will be no access to site-specific data from the central server or from the central study coordinators or investigators. Access to the local dataset will be logged by the local DELTA server, and will be available to the site s principle investigator (but not to central study coordinators or principle investigator). 2. Access to the de-identified central data repository will be restricted to authorized users of the central server, and will be listed in the central site s IRB application. The system will maintain logs of all user interactions with the data set which will be available to the principle investigator from all participating sites. 3. Data analysis and exploration will be restricted to those analyses required to fulfill the goals of this study. Any additional data analysis will require review and approval of the study oversight committee, which will be comprised of the principle investigators from each participating site as well as the overall study principle investigator. Any additional analyses requested by the study oversight committee, in addition, will require IRB approval at each participating site. 4. The de-identified study database will be preserved for a period of 5 years following completion of enrollment in the study. Any secondary use or analysis of the data will require approval by the study oversight committee and IRB approval from each participating center, as noted in item (3) above. After 5 years, the data at the central server will be destroyed, though the results of all analyses performed will be retained at the coordinating center, for archival purposes. 5. The local study databases, at each participating sites, will be destroyed 90 days after the completion of the study, unless the center requests, in writing, to retain the local DELTA system for internal quality assurance purposes. If the local center retains the DELTA system, software support of the system will continue for 12 months without charge. After that time, specific independent arrangements will need to be made with the software vendor to assure continuing system support. 8

10 References 1. Matheny ME, Arora N, Ohno-Machado L, Resnic FS. Rare adverse event monitoring of medical devices with the use of an automated surveillance tool. AMIA Annu Symp Proc. 2007: PMCID: Matheny ME, Morrow DA, Ohno-Machado L, Cannon CP, Sabatine MS, Resnic FS. Validation of an automated safety surveillance system with prospective, randomized trial data. Med Decis Making. 2009;29(2): PMCID: Matheny ME, Ohno-Machado L, Resnic FS. Monitoring device safety in interventional cardiology. J Am Med Inform Assoc. 2006;13(2): PMCID: Matheny ME, Ohno-Machado L, Resnic FS. Risk-adjusted sequential probability ratio test control chart methods for monitoring operator and institutional mortality rates in interventional cardiology. Am Heart J. 2008;155(1): Resnic FS, Zou KH, Do DV, Apostolakis G, Ohno-Machado L. Exploration of a Bayesian updating methodology to monitor the safety of interventional cardiovascular procedures. Med Decis Making. 2004;24(4): Tiroch KA, Arora N, Matheny ME, Liu C, Lee TC, Resnic FS. Risk predictors of retroperitoneal hemorrhage following percutaneous coronary intervention. Am J Cardiol. 2008;102(11): Resnic FS, Zou KH, Do DV, Apostolakis G, Ohno-Machado L. Exploration of a Bayesian updating methodology to monitor the safety of interventional cardiovascular procedures. Medical Decision Making. 2004;24(4): Resnic FS, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Donnelly S, Normand SLT, Matheny ME. Automated surveillance to detect post-procedure safety signals of approved medical devices. JAMA Nov 10; 304(18): Resnic FS, Preliminary DELTA Network Analysis Study (in preparation) 10. Vidi VD, Matheny ME, Govindarajulu US, Normand SLT, Robbins SL, Agarwal VV, Bangalore S, Resnic FS. Vascular Closure Device Failure in Contemporary Practice. JINT R1: In Press 11. Vlaar PJ, Svilaas T, van der Horst IC, Diercks GFH, Fokkema JL, de Smet BJGL, van den Heuvel AFM, Anthonio RL, Jessurun GA, Tan ES, Suumejier AJH, Zijlstra SF. Cardiac death and reinfarction after 1 year in the Thrombus Aspiration during Percutaneous coronary intervention in Acute myocardial infarction Study (TAPAS): a 1-year follow-up study. Lancet 2008 June 7:371:

11 FDA Vascular Closure Device Safety Signal Verification of the CathPCI DELTA Study Protocol Version: 1.1 December 4, 2015 Principle Investigator: Frederic S. Resnic, MD MSc Coordinating Site: Lahey Clinic Foundation Lahey Hospital and Medical Center Burlington, MA Objectives The objective of VCD Safety Signal Verification Study (VCD Signal Study) is to assess the persistence of the safety signal noted with the Mynx vascular closure device (VCD) as determined by the NCDR CathPCI DELTA safety study ( DELTA Mynx study ). The DELTA Mynx study had identified a significantly increased risk of the composite endpoint of any vascular complication, use of transfusion and local access site bleeding, as well as significantly increased risks of each of the component endpoints for patients receiving a Mynx VCD following percutaneous coronary intervention (PCI) as compared to a propensity matched population of patients having received an alternative active VCD (predominantly AngioSeal or Perclose VCD). The primary data analyzed included PCI cases reported to the NCDR CathPCI registry between 1/1/2011 and 9/30/2013, and was confirmed in multiple patient subgroups, and in a retrospective analysis of the CathPCI dataset to 1/1/2009. In order to confirm (or refute) the persistence of this signal in more contemporary PCI cases the FDA CDRH has requested VCD safety analysis be performed within DELTA using data from 2014-Q Proposed Signal Verification Methods: For the VCD Signal Study, we propose a pre-specified signal verification and communication protocol as detailed below. We will utilize the same endpoints and methods used in the original DELTA Mynx study in order to evaluate the contemporary safety of the Mynx VCD relative to other active VCD commonly used in clinical practice using more recent clinical data from the CathPCI dataset. In addition, prior to execution of the verification analysis, we propose that FDA CDRH document the intended signal communication actions that will be taken based on the results of the verification analysis. Four possible outcomes of the VCD Signal Study are anticipated: 1. Mynx Safety signal verified with similar relative risk as compared with alternative active VCD. 2. Mynx Safety signal not observed in contemporary dataset (or non-significant safety signal identified due to diminishing utilization of device, or narrowing of difference in safety performance of Mynx relative to other active VCD). 3. Mynx Safety signal persists (with statistical significance), but at a lower relative risk as compared with other active VCD. The level that would constitute a meaningfully lower risk (indicating reduced difference in safety) will be specified by FDA CDRH prior to study initiation. 4. Other findings identified. Study Methodology Outcomes of interest will include the composite of: access sites bleeding (CathPCI data element: 8055) and any local vascular complications (8035, 8060, 8070), as well as the rates of transfusion (840), over time, for the Mynx device relative to other VCD following PCI procedures. High risk subgroups 10

12 including patients over the age of 70yrs, women and patient with a documented history of diabetes will also be separately analyzed. We proposed to use a propensity match analysis in which patients exposed to the device of interest (Mynx VCD) between 4/1/14 and 9/30/15 will be compared to similar patients receiving an alternative VCD within 6 months of the case s exposure. The propensity score matched concurrent control population will be developed using a non-parsimonious propensity model including the same covariates as included in the propensity model used in the CathPCI DELTA Mynx study. The propensity model will be re-estimated using six months of data prior to the study period for the signal verification study. Therefore, we will re-generate the propensity model based on PCI patients treated between 10/1/13 through 3/31/14. For the signal verification study, control exposures will be selected from the entire dataset in a 1:1 ratio, within 6 months of the date of the exposed case, using a fixed propensity score probability caliper width of Adverse event rates will be calculated quarterly, in a cumulative fashion, for the propensity score matched cohorts for each device-outcome analysis. The Wilson test evaluating differences in proportions will be used to assess significance with DELTA alert thresholds set to trigger at a p-value < 0.05, after correction for multiple comparisons based on the number of calendar quarters analyzed (6 quarters of data, representing CathPCI cases performed between 4/1/14 and 9/30/15). The data analysis will be performed within the DELTA application, and will be available for review to authorized users of the system. The DELTA analysis, including primary and pre-specified high risk subgroups will be performed within 10 days of the uploading of the contemporary CathPCI data to the DELTA server (housed at NCDR), and will be presented within 4 weeks of data upload to FDA. It is our goal that the final study manuscript will be submitted for publication within 2 weeks of submission of this analysis to FDA. Human Subjects Institutional Review The study protocol, and all other study related materials has been reviewed by the Lahey Hospital and Medical Center institutional review board (IRB), and a waiver has been granted (letter on file). Project Funding: This research will be supported through research grants from the U.S. Food and Drug Administration (HHSF Contract C), the William M. Wood Foundation and the Comparative Effectiveness Research Institute of Lahey Health, Inc. 11

13 Summary of Changes to Study Protocol and Statistical Analysis Plan A. Pre-specified Analysis Plan: Several methodological details were not explicitly specified in the original protocol but were required prior to initiation of the analysis within DELTA and are described in detail below. A.1. Exclusion criteria: Based on prior clinical research on VCD performance, the steering committee chose to exclude all patients who received more than one VCD, those patients who had any non-femoral access, those patients who had multiple sites of access and those patients who received large caliber intravascular devices such as intra-aortic balloon pumps or ventricular support device as part of the PCI procedure. These exclusions were made in an attempt to isolate those patients exposed to a single active VCD, who would have been potential candidates to receive any other active VCD. A.2. Variable selection criteria for propensity match model: The initial covariate selection was based on prior publications related to comparative safety of VCD using propensity matching. These variables were selected from among the CathPCI Registry data elements if had previously been demonstrated in published studies to be associated with any of the vascular outcomes (including access site injury, bleeding or need for post-procedural transfusion) and the covariate would have been available to the treating physician before the deployment of the VCD. Additional covariates were considered if they were thought to be related to the physician decision to choose one VCD versus another and also related to the outcome. A total of 36 covariates from among the CathPCI Registry pre- and intra-procedure data elements were considered as candidate variables for inclusion in the model. All candidate covariates were then assessed through a data validation step within DELTA using data from the calendar quarter prior to the start of the monitoring period. To guard against the possibility of the model being unable to converge due to quasi-complete (or complete) separation, we calculated the linear Variance Inflation Factor (VIF) for each candidate covariate excluding from the model any covariate with VIF>8, and further assessing any covariate with a VIF >4. For this latter group, we reviewed correlation to identify those covariates that were highly correlated (with correlation coefficient > 0.80 and eliminated one of the two highly correlated covariates. For any pair of highly correlated covariates, we retained the covariate most similar to previously identified risk factors, or the one most intuitively related to bleeding risk. For example, the covariates STEMI on presentation and Emergent Procedure were highly correlated (with STEMI VIF>4 and correlation coefficient of 0.834). We chose to eliminate STEMI and retain Emergent Procedure, as the latter was less correlated with other included variables (including NSTEMI on presentation). In addition to guarding against co-linearity of covariates, the number of variables were limited to prevent potential instability in the propensity model through the inclusion of too many covariates. While propensity models are generally more resilient to over-fitting than traditional risk prediction models, a general principle of including one covariate for every four or greater outcome events has been recommended. However, we chose to not explore the CathPCI data for rates of adverse outcomes, prior to the development of the propensity model, 12

14 in an effort to establish a more generalizable approach to prospective, active, surveillance. In the absence of having accurate adverse outcome rates to guide the limit of covariates to include in the propensity model, we chose to be conservative in the total number of covariates included in the final propensity model, which influenced our original candidate covariate list. A.3. Correction for Multiple Comparisons using O Brien-Fleming Method: As noted in the Final Protocol (5/25/12), in an attempt to reduce the risk of Type I error, an adjustment for multiple comparisons was proposed, but the specific methodologic approach was not specified. After review by the Scientific Steering Committee, we adapted the O Brien-Fleming method for application to the pre-specified, active surveillance analyses. The O Brien-Fleming method spends most of the alpha in the ending time period and thereby is less sensitive to differences in outcomes between device cohorts in early analysis periods. This is beneficial within active surveillance to minimize early, voluminous alerting when the volume of information is low relative to the desired final analysis. B. Additional Changes to Protocol: The approved protocol (5/25/12) included proposed analyses that were performed, but will be reported separately from the current study including the comparative safety analysis of mechanical thrombectomy catheters. In addition, the final approved protocol specified Brigham and Women s Hospital (Boston, MA) as the coordinating center for the study. However, in July, 2012, the coordinating center moved to Lahey Hospital and Medical Center (Burlington, MA). C. Attestation to Adherence to Protocol and Analysis Plan: The authors attest to the fidelity of adhering to the written protocol as provided, as well as the description of which methods and analyses were selected prior to reviewing any results, and finally which analyses were post-hoc in nature. Importantly, and in accordance with the data use agreement between Lahey Hospital and Medical Center and The NCDR, no additional exploratory analyses were performed for this study. 13