Patricia Flatley Brennan, RN, PhD

Transcription

1 The role of the National Library of Medicine in Reimagining the Research-Practice Relationship in the Post-Meaningful Use Era Patricia Flatley Brennan, RN, PhD Director National Library of Medicine With the support of Deborah A. Zarin, MD Director, National Library of Medicine

2 Drawing from our history Celebrating our present... Anticipating our future.

3 The right information at the right time requires Knowing the person Supporting the Science Presenting the knowledge

4 Research-Practice Relationship Research at the point of clinical care Identifying relevant clinical trials Refer patients as participants Capture data that is relevant to trials Clinical care at the point of research Identifying participants and aiding accrual Capturing information relevant to a trial Contribute observations to the research enterprise 4

5 NLM, and reimagining the research-practice relationship 5

6 Federal regulatory innovation stimuli FDAAA Update 6

7 Food & Drug Administration Amendments Act Time Line Sept 2007 FDAAA Enacted Dec 2007 Expanded Registration Required Feb 2008 NLM BoR Working Group on Clinical Trials Meeting Aug 2008 Results Database launched Sept 2008 Basic Results Submission Required Apr 2009 FDAAA Public Meeting at NIH Sept 2009 Adverse Events Information Required Nov 2014 FDAAA Proposed Rule (NPRM) Issued Mar 2015 End of Public Comment Period FALL 2016 Final Rule Published 7

8 8

9 Federal Rulemaking Process Consistent with the Administrative Procedures Act [5 U.S.C. 533] Announce in Unified Agenda Departmental review OMB review (Aug 2016) Agency develops draft NPRM Agency response to comments Publish final rule Departmental (HHS) review Public comment period (ended Mar 2015) Congressional review We are here OMB review Publish in Federal Register (Nov 2014) Rule takes effect

10 STAT News December 13, 2015 Assessed whether institutions reported results and whether they were reported on time Analysis included trials of unapproved drugs or devices (if a certification was not on file) The worst offenders included four of the top 10 recipients of federal medical research funding from the National Institutes of Health: Stanford, the University of Pennsylvania, the University of Pittsburgh, and the University of California, San Diego. 10

11 Enhancing Clinical Trial Transparency Under the law, it says you must report. If you don t report, the law says you shouldn t get funding. I m going to find out if it s true [that the research centers aren t reporting the results] and if it s true, I m going to cut funding. That s a promise. Vice President Joe Biden June 29,

12 Why Do We Care? 12

13 Evidence of Problems with the Clinical Research Enterprise (CRE) One practical problem: Potential participants had trouble finding trials. Three scientific problems: Not all trials are published Not all outcome measures (or adverse events) are published Changes to protocols are not always acknowledged 13

14 Journals Do Not Reflect True State of Evidence Paxil GSK suppressed evidence on harms and lack of efficacy in children Vioxx Merck failed to report heart attacks Celebrex Pfizer reported misleading results 14

15 Kaplan-Meier estimates for ulcer complications according to traditional definition. Results are truncated after 12 months, no ulcer complications occurred after this period. (Adapted from Lu 2001.) Jüni P, Rutjes AW, Dieppe PA. BMJ Jun 1;324(7349):

16 16

17 Traditional System Design and conduct of clinical trials was left to individual investigators No specific training required Not much oversight of analytic methods E.g., absence of scientific review of protocols at institutional level Individual investigators decided whether, when and how to disseminate results of clinical trials Institutions assumed that fundamental academic incentives would ensure that this would happen; But widespread appreciation that not all results would be reported 17

18 Still a problem: You only have to register if you re planning to publish We never planned to analyze those data [arm] FDAAA will never be enforced All of our investigators publish Some studies aren t designed to produce meaningful results, so it would be misleading. You ask for data that we don t have We don t have enough statistical help The data are archived We can t afford to support this effort 18

19 Reasons to Register Clinical Trials and Report Results Human Subject Protections Allows potential participants to find studies Assists ethical review boards and others to determine appropriateness of studies being reviewed (e.g., harms, benefits, redundancy) Promote fulfillment of ethical responsibility to human volunteers research contributes to medical knowledge Research Integrity Facilitates tracking of protocol changes Increases transparency of research enterprise Evidence Based Medicine Facilitates tracking of studies and outcome measures Allows for more complete identification of relevant studies Allocation of Resources Promotes more efficient allocation of resources 19

20 Schematic depicting the functions of the three key components of the Trial Reporting System [TRS]. Zarin DA, Tse T. PLoS Med

21 Content of a Record (Minimum Information Requirements) Registration section Submitted at trial initiation Summarizes information from trial protocol: e.g., Condition Interventions Study Design Includes recruitment information (e.g., eligibility, locations) Results section Submitted after trial completion Summarizes trial results Participant flow Baseline characteristics Outcome measures (including statistical analyses) Adverse events 21

22 General Review Criteria Protocol and results must be clear and informative Review focuses on: Logic and internal consistency Apparent validity Meaningful entries Formatting, including appropriate use of database structure Differs in important ways from peer review Review Criteria: 22

23 Examples of Incoherent Entries mean hours sleep/day time to survival 36 eyeballs in study of 14 people mean time to seizure = 18 people first occurrence of all cause mortality (adjudicated) 200 people analyzed for POM when only 100 started the study 23

24 24

25 Other Linkages 25

26 : Informational Scaffold Journal publications Results database entries Conference abstracts CSRs IPD Record Full protocols SAPs Other study documents Other Information (e.g., press releases, news articles, editorials) 26

27 Number of Registered Studies and Rate of Registration Since Launch of per week ICMJE Policy Effective per week FDAAA Enacted per week 27

28 Number and Registered Studies with Posted Results and Rate of Submission Since Launch of Results Database 100 per week 50 per week 28

29 Source: Wedzicha JA, et al. N Engl J Med Jun 9;374(23):

30 Wedzicha et al. (2016) online at Online publication of a NEJM original article reporting the results of the FLAME Trial (NCT ) explicitly linked to results information posted on : The protocol includes a list of 27 secondary outcome measures; we report data for 19 of these outcomes here and in Sections 4 and 5 in the Supplementary Appendix. The outcomes for which data are not reported herein can be found at ( show/results/nct ). Source: 30

31 31

32 entries compared to published data Sample Key Discrepancies Hartung et al. (2014) Becker et al. (2014) Phase 3 & 4 trials with results on Clinicaltrials.gov & journal publication Trials with results on & highimpact journal publication POM Descriptions 15% 15% POM Values 20% 16% SAEs 35% (Frequent underreporting or omissions in publication) Other AEs 37% (Among 1 AE reported on ) 39% (Frequent underreporting or omissions in publication) 48% (Among all trials) 32 Becker & Ross. Unpublished manuscript; Hartung et al. Ann Intern Med. 2014:477-83; Becker et al. JAMA. 2014:

33 33

34 Conclusions: Our results highlight the need to search for both unpublished and published trials. Trial results, especially serious adverse events, are more completely reported at than in the published article. 34

35 Apps 35

36 36

37 Variety of Challenges Improving usability for wide variety of users Data providers and sponsors Full range of users of public site Confusion about goals Drug code names Living in regulatory environment 37

38 What is our Goal? 1. Make it as straightforward as possible for the motivated person with the requisite knowledge to enter results 2. Make it as clear as possible what the requisite knowledge is: a) Clinical research knowledge and skills b) Understanding of specific trial, and access to summary data 39

39 Reactions to 1-to-1 Results Navigation The session with clinicaltrials.gov was highly effective, and indeed we worked out methods for reporting side effects for crossover studies. I've been very impressed with how responsive the PRS team has been with my questions. Additionally, the quality and completeness of the answers has been fantastic. We so appreciate the time and the discussion yesterday and especially your continued interest in helping us seek the best possible resolution with our challenges with the results section of this study. Thank you so much for your clear explanation. It helps me a lot. I cannot thank you enough for helping me work through this. I have learned a lot, and have saved all the informational links for future use with the site. 40

40 Our Metric of Success for Public Site Optimize accuracy of retrieval Metric of relevance: false negative/false positive study records retrieved in response to user queries Depends on: Complete and accurate study information and details of search processes Facilitate full range of uses of data NOT #s of clicks, hits, page views, unique visitors Some users have very high impact (e.g., journal editor) Archival source of data completeness and accuracy are key NOT rates of accrual for specific trials NOT responsible for existence of bad studies 41

41 But, maintained by the National Institutes of Health, says nothing about any charges to participants. The 16-year-old website is the most comprehensive such database publicly available in the United States, with listings for nearly 220,000 clinical studies in the United States and abroad. It does not independently vet those listings, however, or require trial sponsors to disclose potential costs. I went back to [] and looked at the study again. It doesn t say that patients are the funding source, Smith said. The final affront was the screener s suggestion that she could raise the $14,000 through an online GoFundMe campaign. I was disappointed in the NIH. I thought, Why are you letting this occur? 42

42 Questions to Ask What is being studied? Why do researchers believe the intervention being tested might be effective? Why might it not be effective? Has it been tested before? What are the possible interventions that I might receive during the trial?... How long will the study last? Who will pay for my participation? Will I be reimbursed for other expenses? What type of long-term follow-up care is part of this trial? 43

43 On Monday, the FDA opened two days of hearings on draft guidelines intended to clarify the agency s views on stem-cell treatments. The guidelines, which deal with decade-old regulations, have set off an uproar in parts of the stem-cell world because they could jeopardize many of the clinics operations.... Stem-cell therapies, like other treatments, generally must be tested in clinical trials and approved by the FDA before being marketed to patients. But such clearance is not required if the stem cells are not altered much, are used in a function similar to their original role in the body, aren t combined with another article, and don t have a systemic effect on the body. But the FDA s attempts to clarify the details have been fraught with disagreement. 44

44 Company Specific Drug Code Names Drug/chemical studied remains hidden unless have decoder ring e.g., Gleevec, imatinib, sti 1571, Glivec, CGP-57148, UNII-BKJ8M8G5HI No central archive of identifiers Not assured of 1:1 relationship between name and drug Example: AT1001 specified in records for two drugs studied by two different sponsors 1. larazotide acetate for celiac disease by Alba Therapeutics 2. migalastat hydrochloride for Fabry disease by Amicus Therapeutics 45

45 Life in the regulatory bubble High stakes for all stakeholders (regulated community, public, regulator) Need for rigorous documentation/audit trail Increased transparency leads to greater scrutiny of internal processes Need for staff training and standardization Subject to unending concentric circles of review Tensions between evolving scientific/technical standards or user needs and legal boundaries 46

46 The Final Rule 47

47 How to read the final rule: What decisions were made with regard to scope? What decisions were made with regard to additional data elements? What decisions were made with regard to quality control processes? What decisions were made with regard to other possible requirements? Narrative summaries Full protocols 48

48 Public Comments in Response to Notice of Proposed Rulemaking (NPRM) Comment period: Nov 21, 2014 March 23, 2015 Received ~900 public comments ~ 110 substantive multi-issue comments Stakeholder groups Citizens & patients/patient advocates Investigators, academic institutions, & scientific/professional societies Industry & trade groups journal editors 49

49 Goals of Newly Issued Final Rule on FDAAA and Final NIH Policy on Clinical Trial Reporting Final Rule (FDAAA) To clarify legal requirements for regulated community What is an Applicable Clinical Trial? Who is the Responsible Party? To interpret ambiguous key statutory provisions To convey HHS decisions about additional reporting requirements needed to further the statutory goals Require results information submission for trials of unapproved products Not require narrative summaries Final NIH Policy To require registration and results information submission for all clinical trials funded wholly or partially by NIH, whether or not subject to the final rule 50

50 Key Clinical Trial Reporting Requirements Reporting Requirement ICMJE Policy (Effective in 2005) FDAAA Final Rule (Issued in 2016) Scope Registration Registration & Results Reporting Phase All Not Phase 1 All Intervention Type Funding Source All Drugs, Biologics, & Devices regulated by the FDA Any Any NIH Enforcement Refusal to publish Criminal and civil penalties (up to $10,000/day); Loss of HHS funding Final NIH Policy (Issued in 2016) Registration & Results Reporting All (e.g., including behavioral interventions) Loss of NIH funding 51

51 52

52 53

53 4 Scientific Modules Participant Flow Baseline Characteristics Outcome Measures Adverse Events Other, including Certain Agreements 54

54 Results: Baseline Characteristics Publication ( Table 1 ) Baseline Measures Placebo + Rituximab + Total Prednisone Prednisone Number of Participants Age [units: years] 40.5 ± ± ± 11.9 Mean ± Standard Deviation Gender [units: participants] Female Male Race [units: participants] White African American Hispanic Asian/Pacific Islander Other Disease duration [units: years] Mean ± Standard Deviation 8.7 ± ± ± 7.3 Adapted from Merrill JT et al. Arthrit Rheum 2010 and NCT

55 Results: Outcome Measures Publication At week 52, no difference was noted in major clinical responses or partial clinical responses between the placebo group (15.9% had a major clinical response ) and the rituximab group (12.4% had a major clinical response ) Primary Outcome Measure Title Measure Description Participants Achieving Either a Major Clinical Response (MCR) or Partial Clinical Response (PCR) Defined by British Isles Lupus Assessment Group (BILAG) Scores Over the 52-week Treatment Period The BILAG Index is used for measuring clinical disease activity in Systemic Lupus Time Frame Baseline to 52 weeks Measured Values Number of Participants Analyzed [units: participants] Placebo + Prednisone Rituximab + Prednisone MCR (excluding PCR) Figure 2A. Proportion of patients experiencing a major clinical response (MCR) at 52 weeks Adapted from Merrill JT et al. Arthrit Rheum 2010 and NCT PCR Nonclinical Response

56 In Sum:, an NLM invention, is now central to the CRE Reporting required for majority of trials at most AMCs In the new world order, organizations that sponsor studies will be held responsible for their conduct and reporting This will require fundamental changes throughout the CRE: funders, sponsors, investigators The time to decide if study is worth reporting is BEFORE the participants are put at risk, not AFTER 57

57 Papers & monographs Overall Context/Conceptual Framework The Relationship among Registration, Results Reporting, the Clinical Research Enterprise (CRE), and the Trial Reporting System (TRS) Zarin DA, Tse T. Sharing individual participant data (IPD) within the context of the trial reporting system (TRS). PLoS Med Jan 19;13(1):e Zarin DA, Tse T. Moving towards transparency of clinical trials. Science. 2008;319(5868): History of the Registry, Ongoing Challenges, and Case Study: Using Registry Data to Assess the CRE Zarin DA, Ide NC, Tse T, Harlan WR, West JC, Lindberg DA. Issues in the registration of clinical trials. JAMA. 2007;297(19): Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. Characteristics of clinical trials registered in, JAMA. 2012;307(17): History o f the Results Database and Case Study: Using the Results DB to Identify Discrepancies in Peer-Reviewed Publications Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The results database update and key issues. N Engl J Med. 2011;364(9): Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M. Reporting discrepancies between the results database and peer-reviewed publications. Ann Intern Med Apr 1;160(7): Other selected publications by staff available at 58