Submitting Summary Level Site Data: Overview, Implementation, and Case Studies

Transcription

1 Paper RG04 Submitting Summary Level Site Data: Overview, Implementatin, and Case Studies Rb Wlsn, Rh Inc., Chapel Hill, NC USA Jeffrey Ablafia, Rh Inc., Chapel Hill, NC USA Ben Vaughn, Rh Inc., Chapel Hill, NC USA ABSTRACT In December 2012 the US FDA issued draft guidance fr submitting Biresearch Mnitring (BIMO) Summary Level Site Data fr the Center fr Drug Evaluatin and Research s (CDER) Inspectin Planning. The guidance calls fr submitting aggregate results-level data fr each site enrlling subjects in pivtal trials in a clinical develpment prgram, in additin t the subject-level data we currently submit. The FDA is beginning t request this data and the time and resurces required t create them are nt trivial (3). In the past year ur rganizatin has received several requests frm the FDA fr Summary Level Site Data (BIMO dataset). This paper presents an verview f the BIMO dataset and implementatin issues that arise frm its creatin. The paper divides the discussin int fur majr sectins. The first sectin prvides an verview f the guidance pertaining t BIMO data. The secnd sectin describes the input surces and prcesses that are required t create the BIMO dataset. The third sectin presents tw case studies utlining ur experiences and lessns learned. The last sectin prvides a few recmmendatins related t clarifying sme elements f the BIMO dataset and supprting infrmatin. INTRODUCTION In 2012 the FDA Safety and Innvatin Act (FDASIA) and the fifth reauthrizatin f the Prescriptin Drug User Fee Act (PDUFA V) were authrized by the US Cngress. One f the key perfrmance gals f PDUFA V was enhancing benefit-risk assessment fr marketing applicatins. As a result, CDER recently initiated a pilt prgram t evaluate a risk-based mdeling apprach t selecting clinical investigatr sites fr inspectin. This apprach requires that CDER have sufficient data t select sites fr inspectin. In December 2012 the US FDA issued tw draft guidances that facilitate prviding CDER the data required fr the timely selectin f inspectin sites. The first guidance, titled Prviding Submissins in Electrnic Frmat Summary Level Clinical Site Data fr CDER s Inspectin Planning (1) utlines the FDA s recmmendatin that spnsrs submit summary level clinical site data in standardized frmat. The secnd guidance, titled Specificatins fr Preparing and Submitting Summary Level Clinical Site Data fr CDER s Inspectin Planning (2) prvides the technical specificatins and a data mdel fr creating the BIMO dataset. These guidances apply t new drug applicatins (NDAs), bilgics licensing applicatins (BLAs), and NDA and BLA supplemental applicatins cntaining new clinical study reprts that are submitted t CDER. This data mdel is unique in that it supprts the presentatin f aggregate results-level data fr each site in a clinical develpment prgram rather than subject-level data. Cnstructing this dataset can add significant time and cst t preparing the data package that accmpanies a regulatry submissin (3). In the fllwing sectins we lk at the structure and cntent f the BIMO dataset, the inputs and prcesses t create this dataset, case studies and lessns learned, shrtcmings f the data mdel, and recmmendatins fr imprving the mdel. 1

2 BIORESEARCH MONITORING (BIMO) SUMMARY LEVEL CLINICAL SITE DATA DESCRIPTION The technical guidance (2) calls fr submitting a single dataset that includes data frm all pivtal studies used t supprt safety and efficacy in an applicatin. The dataset shuld be named clinsite.xpt and shuld be submitted in SAS transprt file frmat. The structure f the dataset is ne recrd per pivtal study, clinical site, treatment arm, and primary endpint fr the intent-t-treat (ITT) ppulatin. The technical guidance prvides a standard data mdel fr cnstructing this dataset. It includes a list f variables which must be included, their attributes, cntrlled terminlgy, and the metadata required t dcument each variable. The specificatin cntains variables t 1) identify and characterize each study; 2) identify the spnsr; 3) dcument the NDA r BLA number; 4) describe individual clinical investigatr sites, including financial disclsure infrmatin; 5) identify treatment arm; 6) dcument endpint types; and 7) reprt primary endpint results by site and treatment arm. SUBMISSION CONSIDERATIONS Fr applicatins submitted in the ectd frmat, the BIMO dataset shuld be placed in Mdule 5 as specified belw: It shuld be accmpanied with a data definitin file r define file in pdf frmat. The technical guidance includes a sample BIMO dataset as well as a sample define file. Since the impetus fr submitting the BIMO dataset is a mre timely apprach t site selectin, spnsrs shuld carefully cnsider several ptins fr when t submit this data. The ptimal strategy wuld be t prvide the BIMO dataset fr a pre-nda r pre-bla meeting, and ask the FDA if it agrees with the frmat and cntent. If this isn t feasible, a secnd ptin wuld be t submit the BIMO dataset as sn as pssible after the pre-nda r pre-bla meeting. Lastly, ne may wait until the time f the submissin t prvide the dataset. In all cases, the BIMO dataset shuld be included with the NDA/BLA submissin at the path nted abve. If any f the studies have data that present issues, we strngly recmmend discussing the BIMO dataset with the FDA in a pre-submissin meeting. INPUT SOURCES AND PROCESSES ONE DATASET MANY GOALS, MANY INPUT SOURCES The risk-based mdel mentined abve uses the BIMO dataset as input. The BIMO dataset is intended t describe and summarize the characteristics and utcmes f clinical investigatins at the individual study site level acrss a clinical develpment prgram. Mre specifically: The summary level clinical site dataset is intended t characterize the individual clinical investigatr sites, describe aspects f the studies with which thse clinical investigatr sites are assciated, and present the characteristics and utcmes f the study at the site level. (1) Given the wide array f data that are presented in a single lcatin, it is nt surprising that many surces f data are used t create the dataset. The surces f data include, but may nt be limited t: SDTM (r raw) datasets frm multiple studies ADaM (r legacy analysis) datasets frm multiple studies clinical trials management system (CTMS) utput financial disclsure frms and databases, and regulatry affairs databases (2,3). The infrmatin requested in the BIMO dataset and the likely rigin f each are summarized in Table 1. 2

3 TABLE 1: INFORMATION REQUESTED IN BIMO DATASET AND THE (LIKELY) DATA SOURCE/ORIGIN Study-level metadata Data Surce/Origin Infrmatin Cmment(s) Study Number; Title, Number f Spnsrs; Spnsr Name(s); IND Number; Study Cnducted Under IND? Regulatry affairs database Raw/SDTM/Legacy Analysis/ADaM dataset(s) Prtcl/Statistical Analysis Plan NDA Number; BLA Number; Supplement Number Site Identifier; Treatment; Number f Subjects Enrlled; Subjects Screened; Subjects Discntinued; Treatment Efficacy Endpint Estimate; Treatment Efficacy Endpint; Site-specific Treatment Effect (and variatin); Number f Censred Observatins; Number f nn-saes Number f SAEs; Deaths Text descriptin f Primary Endpint(s); Primary Endpint Variable Type. Data are mst likely cntained in ADaM efficacy and safety datasets. Hwever, there are scenaris in which the sitespecific analysis results requested may nt have been cnducted as a part f the planned statistical analysis (see Case Study 1, belw). Nne hardcde per guidance (2) Dmain Abbreviatin Guidance requests that DE be used as the dmain abbreviatin. CTMS (r ther separately maintained database) FDA 1572 frms Number f Prtcl Vilatins Maximum financial disclsure amunt by any single investigatr by site; ttal financial disclsure by site (fr all principal and sub-investigatrs); investigatr names, phne numbers, fax numbers, addresses, and pstal address infrmatin (cuntry, state, prvince, pstal cde, street, etc.) Prtcl vilatin infrmatin is frequently captured in the SDTM DV dataset. Hwever, fr lder legacy studies, that may nt be the case. Data may be cntained in a regulatry affairs database, but ften it is nt. Nt nly are there several data surces, but it is als likely that: the data surces listed abve are maintained by different functinal grups (e.g., regulatry, bistatistics, clinical mnitring); the different functinal grups have different standards fr data accuracy, validatin, and quality cntrl; the functinal grups are a part f different rganizatins (e.g., independent cnsultants, CROs, spnsr,); and/r multiple rganizatins may have verlapping respnsibilities within the same clinical develpment prgram. A prpsed prcess fr assembling and summarizing this infrmatin is detailed in the fllwing sectins. PLAN AHEAD THIS IS NOT YOUR STANDARD STANDARDIZED DATASET As nted abve, there are numerus input surces that are ptentially stred and maintained in varying frmats, with varying levels f accuracy, by several functinal grups. As such, preparing, assembling, summarizing, and verifying the accuracy f the cadre f infrmatin required t create the BIMO dataset is a prcess that shuld be undertaken as early as pssible in marketing applicatin develpment. Based n ur recent experiences, we think sme essential steps t successfully creating a submissin-ready BIMO dataset are: 1. Start the data assembly prcess as early as pssible. Regardless f the steps ne chses t take in creating the BIMO dataset, start the prcess as early as pssible. This is particularly true in large, multi-center, and/r internatinal clinical develpment prgrams. 3

4 Starting the prcess at the time f final site selectin in the pivtal trial(s) is best. Much f the essential regulatry infrmatin is nt nly available at this time, but can als be cnslidated int a digestible frmat fr later prgraming. In additin, this increases the likelihd that the BIMO dataset will be available fr the pre- NDA r pre-bla meeting. 2. Identify and select a BIMO dataset crdinatr. Given the numerus functinal areas (and cmpanies) that may need t cntribute infrmatin t the BIMO dataset, there shuld be a single BIMO dataset crdinatr. 3. Create and maintain a data/infrmatin surce inventry and actin lg. There are mre than 30 data elements included in the BIMO dataset. Creating and maintaining a data/infrmatin surce inventry lg can be useful. Fr each data element and each study, cllecting the fllwing infrmatin is recmmended: Requisite surce data As nted abve, input data can cme frm many surces and many functinal areas. Take an inventry f the data that are needed t create the BIMO dataset elements fr each study. Fr a small clinical develpment prgram, the list f requisite data may be shrt and may be maintained within a single rganizatin. Hwever, this is ften nt the case. The functinal area and wner f the data Identify the grup (and individual) respnsible fr creating and maintaining the data surce(s). In sme cases, the grup respnsible may nt be part f a functinal area accustmed t creating, maintaining, and verifying the accuracy f data t be submitted t the FDA. Establishing cntact early can help get a jump n understanding and, if necessary, updating the current surce data. Surce data status After listing all f the surce data required t create the BIMO dataset, the BIMO dataset crdinatr shuld evaluate the status f each surce data element. In general, the surce data elements fall int ne f the fllwing categries: the surce data are validated and are well-suited fr inclusin in the BIMO dataset (e.g., adverse event data by site) and n additinal validatin r quality cntrl (QC) activities are required; the surce data are validated and are well-suited fr inclusin in the BIMO dataset, hwever, the analyses requested in the BIMO dataset may nt have been previusly perfrmed (e.g., site-level analysis f the primary endpint when the primary analysis des nt supprt by-site treatment estimates); the surce data are in a nn-validated database (e.g., a regulatry affairs database using Excel r Access); r the surce data d nt exist. After cllecting the abve infrmatin, establishing the fllwing elements f a data frmat and/r analysis validatin and QC actin plan is recmmended: Planned data frmat Select an acceptable data frmat fr the data element (e.g., Access, Excel, SAS, etc.). Discussins with the individual(s) respnsible fr prgramming the BIMO dataset can be useful. Required additinal analyses Identify analyses that were nt cnducted as a part f the riginal, planned analyses. 4

5 Prductin, validatin, and QC plan A prductin, validatin, and/r QC plan may need t be established fr data elements that require a new data frmat, are hused in a nn-validated database, r have nt previusly been entered int a database. Prductin, validatin, and QC plan wner Identify the individual respnsible fr verseeing the cmpletin f prductin, validatin, and/r QC activities f the surce data element. A sample f tw BIMO data elements frm a data/infrmatin surce inventry and actin lg is shwn in Table 2: TABLE 2: SAMPLE OF DATA ELEMENTS FROM DATA/INFORMATION SOURCE INVENTORYAND ACTION LOG Variable Name PROTVIOL FINLDISC Variable Label Number f Prtcl Vilatins Disclsure Amunt Descriptin Number f prtcl vilatins at a given site by treatment arm. This value includes multiple vilatins per subject and all vilatin types (i.e., nt limited t nly significant deviatins). Ttal financial disclsure amunt (USD) by site calculated as the sum f disclsures fr the principal investigatr and all sub-investigatrs t include all required parities. Under the applicable regulatins (21 CFR Parts 54, 312, 314, 320, 330, 601, 807, 812, 814, and 860). Study xx-03, xx-04, xx- 05 xx-01, xx-02 xx-03, xx-04, xx- 05 xx-01, xx-02 Functinal Area (Owner) Bistatistics (Persn 1 Cmpany x) Clinical Mnitring (Persn 2 - Cmpany y) Regulatry Affairs (Persn 3 Cmpany z) Regulatry Affairs (Persn 4 - Spnsr) Current Data Frmat Prtcl deviatin data fr studies xx-03 and xx-04 are in the SDTM DV dmains. Spnsr nly cllected and reprted prtcl deviatin data fr legacy studies xx-01 and xx-02 in a CTMS database. Financial disclsure infrmatin has nt been cnslidated int a database. Infrmatin is cntained in scanned.pdf FDA 1572 frms. Financial disclsure infrmatin was keyed int the Excel spreadsheet lcated here (file path). Planned Data Frmat SDTM DV data frm studies xx- 03, xx-04, and xx-05 will be used as input fr the BIMO dataset. N change. SAS dataset Excel spreadsheet N update; Excel spreadsheet frmat can be used as input int BIMO dataset Prductin / Validatin / QC Plan (Owner) Validated via applicable SOPs. (Persn 1 Cmpany x) An interim SAS dataset will be created. CTMS will be merged with existing legacy datasets t btain key subject and site infrmatin. Prductin and validatin will be Financial disclsure infrmatin by site, PI, and sub-pi will be keyed int the Excel spreadsheet template lcated here An independent QC f the keyed values will be cnducted. (Persn 5 Cmpany z) 5

6 TABLE 2: SAMPLE OF DATA ELEMENTS FROM DATA/INFORMATION SOURCE INVENTORYAND ACTION LOG Variable Name PROTVIOL FINLDISC cnducted by statistical prgramming grup. Specificatins fr interim prtcl deviatin dataset are fund here (file path). (Persn 1 Cmpany x). (file path). The keyed values will be independently QC d based n the abbreviated QC plan lcated here (file path). (Persn 3 Cmpany z) Cmpletin Date N/A mm/dd/yyyy mm/dd/yyyy mm/dd/yyyy Status Cmplete Validatin in prgress. 4. Create BIMO dataset prgramming specificatins Data entry in prgress. Check in mm/dd/yyyy. Independent cmparisn f spreadsheet t master site list, PI list, and sub-pi list and t FDA 1572 frms nging. Expected cmpletin date mm/dd/yyyy. Specificatins shuld be created that describe the prgrammatic steps required t pull tgether all f the data surces identified in the inventry and actin lg. 5. Prgram, validate, and QC the BIMO dataset With the previus steps cmpleted, the BIMO dataset can be prgrammed and independently validated. Althugh it is ur experience that the prgramming and independent validatin prcess is similar t that f mst ther standard datasets, the QC prcess is mre invlved. Sme pints f emphasis in the BIMO dataset QC prcess are as fllws: Verificatin f the frmat f the dataset per the guidance (2) are the apprpriate number f recrds included in the dataset (e.g., Number f Studies x Number f Sites x Number f Primary Endpints x Number f Treatment Arms) and are the apprpriate fields ppulated? Cnfirmatin that recrds are assciated crrectly the numerus data surces, data surce frmats, and functinal areas/grups respnsible fr the creatin f the surce data can easily prduce prgramming mistakes. Are the right study/site/endpint recrds frm each data surce crrectly cnnected? Validatin that efficacy and safety-related statistics are calculated crrectly are the statistics cntained in the study-level CSRs (if previusly cmputed) the same as the statistics cntained in the BIMO dataset? If the statistics in the BIMO dataset have nt previusly been cmputed, are they accurate? Regulatry-related QC are all investigatrs included, is the cntact infrmatin crrect, are the financial disclsure amunts crrect, etc.? 6. Prepare fr additinal FDA requests The FDA has cnsistently requested supprtive by-study, by-site patient level listings and ther supprtive data summaries. Spnsrs shuld anticipate additinal FDA requests and allcate resurces t cmply. 6

7 RECENT EXPERIENCES AND CASE STUDIES PhUSE 2014 The fllwing sectins describe tw recent experiences related t BIMO dataset cnstructin, including challenges and a brief discussin. CASE STUDY 1 Clinical Develpment Prgram descriptin: This was a multi-natinal, multi-center develpment prgram including three phase II studies and fur phase III studies. Data were submitted as a part f a 505(b)(1) marketing applicatin. Data surce status: Surce data included raw data, legacy analysis data, SDTM datasets, ADaM datasets, and a regulatry affairs database. In sme cases, data had nt been assembled int a database suitable fr use in prgramming. Challenges: Challenges encuntered while creating the BIMO dataset included the fllwing: Fur different CROs were respnsible fr executing the seven phase II and phase III studies. Data were in several different frmats. N regulatry database existed fr tw f the studies. The primary analysis f the primary endpint utilized an analysis methd which did nt supprt by-site estimates. Althugh the guidance states that nly pivtal studies be included in the BIMO dataset (2), subsequent FDA cmmunicatin requested that phase II and phase III studies be included. A request was made by the FDA t als include 10 different by-study, by-site patient level listings with the BIMO dataset and dcumentatin. Discussin: The primary challenges in this example were: 1) assembling all required data and identifying missing data; 2) incrprating primary analyses results which did nt supprt by-site analyses; and 3) cmplying with updated FDA requests. Data needed t create the BIMO dataset were held by varius CROs, the spnsr, and independent cnsultants. Lcating the requisite data and identifying data that were missing was a time cnsuming prcess, in particular since sme f the studies were cmpleted several years earlier. Sme data were stred in paper files and required manual entry and verificatin. The primary analyses f sme f the studies included in the BIMO dataset did nt supprt by-site treatment estimates. A related, thugh different, analysis apprach was used t create by-site treatment and treatment difference estimates. The different apprach was described in the assciated BIMO dataset dcumentatin. The FDA-requested BIMO patient level listings required additinal prgramming, validatin, and QC time. New listings were created as the listings created t supprt the study-level CSRs did nt match the frmat and cntent f the FDA-requested BIMO listings. CASE STUDY 2 Clinical Develpment Prgram descriptin: This was a multi-center develpment prgram including fur phase III studies. Data were submitted as a part f a 505(b)(2) marketing applicatin. Data surce status: Surce data included SDTM datasets, ADaM datasets, and a regulatry affairs database. Challenges: Challenges encuntered while creating the BIMO dataset included the fllwing: Tw different CROs were respnsible fr executing the phase III studies. The primary analysis f the primary endpint was nn-parametric. 7

8 A request was made by the FDA t als include 10 different by-study, by-site patient level listings with the BIMO dataset and dcumentatin. Discussin: The tw primary challenges in this example were: 1) mapping nn-parametric analysis results int the BIMO dataset specificatins; and 2) creating by-study, by-site patient level listings. At present, the BIMO dataset specificatins assume parametric results, including means, rates, and assciated measures f variability. In an effrt t prvide measures f central tendency related t treatment, median values were prvided. Similarly, in an effrt t characterize treatment difference, the Hdges-Lehman-Sen shift statistic was prvided. Because measures f variability were nt easily applicable t the shift statistic, the assciated p-value was reprted instead. These deviatins frm the BIMO dataset specificatins were included in the assciated dcumentatin. Unlike Case Study 1, the patient level listings used t create the study-level CSRs were already created in a frmat that culd easily be adapted t the frmat requested by the FDA. Thugh in a similar frmat, an extensive table f cntents had t be created fr each study, site, and listing type in rder t cmply with the FDA s request. A FEW RECOMMENDATIONS Althugh the BIMO dataset specificatins and guidelines generally prvide a clear radmap fr its creatin, the fllwing are a few recmmendatins related t clarifying sme elements f the BIMO dataset and supprting infrmatin: Guidelines shuld be created t accmmdate primary analyses that are nn-parametric. If required, supprtive by-study, by-site patient level listings shuld be included as an element in the relevant guidance dcuments. Additinal specificatins shuld be created t clarify expectatins fr: missing r nn-missing values (e.g., sme data elements are site specific, but nt treatment specific); placehlder recrds fr arms that d nt ccur at a given site studies with multiple phases and/r multiple treatment cmparisns; and characterizing safety and efficacy data in cases where the safety and efficacy ppulatins are nt the same (e.g., subject data presented as randmized in efficacy data elements, but presented as treated in safety data elements). BIMO dataset metadata shuld cmply with standardized metadata guidelines. CONCLUSION It is likely that the BIMO dataset will cntinue t be a part f FDA marketing applicatins fr the freseeable future. Cnstructin f the site-level BIMO dataset fr a clinical develpment prgram can be a time cnsuming prcess that ptentially invlves numerus service prviders, functinal areas, and input surces. CROs and ther rganizatins respnsible fr its assembly shuld begin preparing the dataset as early as pssible in the marketing applicatin prcess. Successfully executing all f the activities invlved in BIMO dataset submissin, frm identifying and assembling all f the relevant infrmatin (r perhaps cnfirming its absence) thrugh BIMO dataset prductin and QC, require a well-rganized plan. REFERENCES (1) Guidance fr Industry: Prviding Submissins in Electrnic Frmat Summary Level Clinical Site Data fr CDER s Inspectin Planning (FDA DRAFT Guidance: December 2012). (2) Specificatins fr Preparing and Submitting Summary Level Clinical Site Data fr CDER s Inspectin Planning (FDA DRAFT Guidance: Technical Specificatins, Nvember 2012). (3) Jhnsn, James, Evlving Regulatry Guidance n Submissin f Standardized Data Annual PhUSE Cnference. 8

9 ACKNOWLEDGMENTS We wuld like t thank ur clleagues Carl Baker and Frank DiIri fr their valuable input during preparatin f this paper. SAS and all ther SAS Institute Inc. prduct r service names are registered trademarks r trademarks f SAS Institute Inc. in the USA and ther cuntries. indicates USA registratin. Other brand and prduct names are trademarks f their respective cmpanies. CONTACT INFORMATION Yur cmments and questins are valued and encuraged. Cntact the authrs at: Rb Wlsn Rh, Inc. Rb_wlsn@rhwrld.cm Jeffrey Ablafia Rh, Inc. Jeff_ablafia@rhwrld.cm Ben Vaughn Rh, Inc. Ben_vaughn@rhwrld.cm Brand and prduct names are trademarks f their respective cmpanies. 9