We are looking forward to reading the revised manuscript and, we hope, making a final acceptance decision.

Transcription

1 14-Apr-2016 Dear Mr. Hwang Manuscript ID BMJ entitled "Safety, trial outcomes reporting, and regulatory approval of medical devices in Europe and the United States: cohort study" Thank you for sending us your paper. We are pleased to say that we would like to publish it in the BMJ as long you are willing and able to revise your paper as explained below in the report from the manuscript meeting. We are provisionally offering acceptance but will make the final decision when we see the revised version. The report from the manuscript meeting, the comments from the reviewers and general requirements for submission are available at the end of this letter. We are looking forward to reading the revised manuscript and, we hope, making a final acceptance decision. Please accept my apologies for the delayed decision letter: I could not take part in this manuscript meeting because I had to attend a conference. Yours sincerely Georg Roeggla groggla@bmj.com, **Report from The BMJ s manuscript committee meeting** These comments are an attempt to summarise the discussions at the manuscript meeting. They are not an exact transcript. Manuscript Meeting Committee: Elizabeth Loder (chair), Tim Cole (statistician), Alison Tonks, Wim Weber, Jose Merino, Tiago Villanueva, Jessamy Bagenal, Rubin Minhas (he did not take part in the discussion due to a COI) Decision: Provisional acceptance The committee was interested in the topic of your research. The following issues were discussed: Is a statement of Conformité Européenne is the same as approval? According to medical devices are not approved in the EU. Notified bodies verify the conformity of a product with European regulations. Please clarify. When you say Europe do you actually mean EU or all European countries? Device approval is a complex issue and varies by location. The committee would like to see a box that better describes the differences between jurisdictions. The committee found the methods complicated. Another appendix would help listing the sources these authors had to search to find their devices, and precisely how they did it. Replication isn't possible from this report alone. The title could be clearer. You need to signal that the study compared US and EU regulatory pathways for selected devices. The abstract is hard to follow. The paper could be much simpler with unnecessary detail omitted. NB CE marking not CE mark. You describes 67% as most in several places oversimplification. Figures 2 and 3 informative, analysis in Tables 2 and A2 and A3 overblown. Generalisability looks limited. The RQ is not necessarily logical. First, please revise your paper to respond to all of the comments by the reviewers. Their reports are available at the end of this letter, below. Please also respond to the additional comments by the committee. In your response please provide, point by point, your replies to the comments made by the reviewers and the editors, explaining how you have dealt with them in the paper. Please note that the BMJ might choose to shorten content or replace or re-size images for the print issue. ** Comments from the external peer reviewers** REFEREE COMMENTS Reviewer: 1 Recommendation: Comments: I like this paper and it has a good face validity!

2 a few important issues to address: One of the main findings is that 'The unadjusted rate of safety alerts and recalls for devices approved first in EU was 27% (62/232) versus 14% (11/77) for devices approved first in the US'. There is a stong conclusion based on this. However, this might be related to the threshold to issue alert rather than evidence of safety. To ensure this is correct one should do good systematic review on all products but that would be a huge task. Still we know that FDA is less likely to issue alert and more likely to issue request to investigate the signals-- are these taken into account? also in some instances Europe has better system of registries to generate safety signals-- can you please research all of these? i'm worried that this conclusion will be misleading.. other conclusions are justified. the authors cite a paper on TAVR related to non-operable patients but summarized it wrong-- the citation 6 -- 'for example, transcatheter aortic8valve replacement was developed as an alternative to aortic valve surgery for treatment of symptomatic aortic stenosis, and has been associated with higher rates of survival than surgical aortic8valve replacement' -- please correct this statement. the authors site many examples of device safety issues found recently in this statement 'The recent emergence of safety issues involving implanted devices in Europe9,10,11 and the US12,13,14 has renewed calls to revisit the appropriate trade-offs between device access and risk. I'm surprised they don t cite the analytic papers on metal on metal failure in BMJ, lancet and editorials for them highlighting the failures of regulation and policy. there is a statement about 'The European Society of Cardiology and others have called for a centralized system for evaluating high8risk devices; stronger and more transparent clinical data requirements that incorporate expert medical advice; and public education on the limitations of the CE mark6. We advocate for registry of innovations within IDEAL initiative. I can supply reference for this. the IDEAL is international leader in promoting publishing failed innovations and should be cited-- the statement 'Given poor trial publication rates, patients and clinicians need greater regulatory ransparency to make informed treatment decisions' the most important conclusion in my opinion. Additional Questions: Please enter your name: art sedrakyan Job Title: Professor of Healthcare Policy and Research Institution: Weill Cornell Medical College Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests (please see BMJ policy) please declare them here: Reviewer: 2 Recommendation: Comments: This is an important study. Since there is no publicly accessible list of medical devices in Europe, the investigators had to search multiple databases and sources to identify new medical devices that received approval over a 6-year period; they succeeded in retrieving data about more than 300. They report, for the first time from a large series unlike previous anecdotal reports, the time lag between approval of devices in the European Union and the USA. Crucially, they observed that a higher proportion of devices first approved in Europe was withdrawn or subject to safety alerts, compared with devices first approved in the USA. This suggests that

3 European patients were indeed exposed to additional risks compared with American patients, as a consequence of the regulatory system. These observations are timely, given the current status of negotiations on the new Draft Regulation on medical devices in the EU. The paramount need for transparency of clinical data relating to new medical devices is an essential message for legislators in the European Council and Parliament. The finding that only 37% of pivotal trials are published within 5 years of approval, strongly emphasises the need for much tighter conditions for post-market surveillance and continued market access. The study appears to have been well designed, conducted, and reported. I have no concerns about the results, although it would be interesting to know if the investigators collected the details of which notified bodies approved the devices that they included an analysis of recalls and alerts in relation to the size, location, and experience of the notified bodies would be interesting (perhaps as the basis of a separate report?). The conclusions are entirely appropriate. Specific comments: Abstract/Setting (p 2, line 12): As I understand it, devices are not approved in Europe by getting the CE mark. Strictly speaking, the application by the manufacturer is approved by a Notified Body, or in the case of some high-risk devices after review of clinical trial data also by the national regulatory agency (in which case the national regulatory agency or competent authority will also have approved the trial design). Approval by NB or CA then allows the manufacturer to affix the CE mark. If this nuance is unclear it might be wise to have a regulator review the format of words in the manuscript. Page 3/Study snapshot: It is not true to state (as a blanket assertion) that in the EU medical devices do not require evidence of effectiveness ; some do. The general test is safety and performance, with positive risk/benefit, but sometimes for high-risk devices, trials are expected. Pivotal trials, however, are more often performed in the US (as stated). Page 4/line 32: Reference 10 relating to PIP breast implants is not a good example of failure of requirements for approval; it was a failure of regulatory surveillance to discover fraud on the part of an unscrupulous manufacturer. The subsequent Joint Action Plan has led to much greater supervision of notified bodies and manufacturers. This could be mentioned at the end of this paragraph together with a more specific comment (new draft regulation now in late stages of negotiation; or something similar). There is an official device database in Europe ( Eudamed ) but currently it seems not to function well or be used much. One of the provisions in the new law is that this should be greatly developed. The name might be mentioned in the manuscript (for example, paragraph 2 on page 11). Page 16, box 1: The European summary is not entirely accurate. Notified Bodies are notified by their national competent authority (regulatory agency) for specific sectors or legislation (for example, 16 for active implantable medical devices), and supervised by them, so they are not in some senses totally independent. The number of NBs for legislation in all industrial sectors is >2000, but those approved for medical devices is about 60 (after some were denotified as a consequence of the Joint Action Plan). The paragraph as written implies that the 60 NBs evaluate products from all sectors, which is almost certainly untrue. Competent authorities do apply some uniform standards, and there will be many more following the new EU Regulation. Devices in Europe are evaluated according to their level of risk, just as in the USA. The details of all NBs are available at the NANDO database maintained by the EU: Page 22, appendix table 1: the term Cardiac assist devices is not very clear; it sounds as if it might include artificial hearts or circulatory support. Could the offers clarify this? Page 26, appendix protocol: this text reads as if it has been transferred from an application to perform the study, rather than formulated as a report of the study. Please change we propose to.. etc to the past tense. Alan Fraser Additional Questions: Please enter your name: Alan Fraser Job Title: Professor of Cardiology Institution: Cardiff University Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: No Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

4 Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No If you have any competing interests (please see BMJ policy) please declare them here: No competing interests but I serve on several committees on medical devices of the European Commission. Reviewer: 3 Recommendation: Comments: The manuscript provides an important addition to the literature regarding key safety and regulatory issues for medical devices approved in the EU and the US. I strongly recommend accepting it for publication after a few of the findings are clarified. The article would be substantially strengthened with better clarity regarding the presentation of results. Specific comments and suggestions are detailed below: 1. Page 5, Line 11-12: Why were cardiovascular, neurologic, and orthopedic devices selected for study? This is justified briefly at the end of the article, but why not at the beginning? Were these most likely to be high profile devices and have data available? Were other specialties (e.g anesthesiology, surgery, gastroenterology/urology) considered? Additional clarification/support at the start or end of the article would be useful. 2. Page 5, Line 49: Small and medium-sized companies are classified as having gross revenues less than US$1 billion, but Figure 2 and Table 2 only show Small (=US$1B) companies. Can we assume that medium sized companies in the figure/table are lumped with the small group? Updating either the text or table/figure to make the company classifications consistent would be helpful. 3. Page 7, Line 36-8 ( Of the 75 devices categorized as major innovations, a greater proportion ): While the 72% and 74% mentioned seem reasonable, it is unclear how 20/75 (27%) of devices approved through the PMA pathway would constitute a greater proportion? Even if the denominator is 54 rather than 75, this is still not a majority. Clarification or rephrasing is needed. 3. Page 7, line 42 Presentation of data are confusing Rather than having a medium difference in approval time for all devices combined, it would make more sense to separately analyze the 510(k) vs, the non 510(k) devices, since it seems that the 510(k) devices are cleared more quickly in the US than in the EU, but all the others are approved more slowly in the US. Another alternative would be to also look at the compassionate use devices separately, because there are so few of them and the standards are different. Either way, within that group of non-510(k) devices, keep the analysis of each type of US approval pathway separately, as was done. 4. In the presentation of the results, there is so much confounding between whether a device is a major innovation and whether it was approved through the PMA, that it seems confusing to look at those variables separately. But if they are looked at separately, it would be helpful to also look at both variables together, e.g. the results for PMA devices that were also major innovations vs PMA devices that were not major innovations, vs. non-innovative devices that were PMA vs. non-innovative devices that were not PMA. These variables have implications for safety alerts, delays in publication of data, etc. 5. Page 10, Line 23-25: As noted above, the fact that the chosen specialties represent a majority of clinically relevant devices in the U.S. regarding patient safety issues should be mentioned earlier to explain a valid motivation for choosing these device types. Additional Questions: Please enter your name: Diana Zuckerman Job Title: president Institution: National Center for Health Research Reimbursement for attending a symposium?: No A fee for speaking?: No A fee for organising education?: No Funds for research?: Funds for a member of staff?: No Fees for consulting?: No Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

5 Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: Yes If you have any competing interests (please see BMJ policy) please declare them here: Publication of this paper could indirectly lower the value of my stock in J & J. But that's true of most of the work I do. **Information for submitting a revision** Deadline: Your revised manuscript should be returned within one month. How to submit your revised article: Log into and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision. You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Once the revised manuscript is prepared, you can upload it and submit it through your Author Center. When submitting your revised manuscript, you will be able to respond to the comments made by the reviewer(s) and Committee in the space provided. You can use this space to document any changes you make to the original manuscript and to explain your responses. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewer(s). As well as submitting your revised manuscript, we also require a copy of the manuscript with changes highlighted. Please upload this as a supplemental file with file designation Revised Manuscript Marked copy. Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission. When you revise and return your manuscript, please take note of all the following points about revising your article. Even if an item, such as a competing interests statement, was present and correct in the original draft of your paper, please check that it has not slipped out during revision. Please include these items in the revised manuscript to comply with BMJ style (see: and Items to include with your revision (see 1. What this paper adds/what is already known box (as described at 2. Name of the ethics committee or IRB, ID# of the approval, and a statement that participants gave informed consent before taking part. If ethics committee approval was not required, please state so clearly and explain the reasons why (see 3. Patient confidentiality forms when appropriate (see 4. Competing interests statement (see 5. Contributorship statement+ guarantor (see 6. Transparency statement: (see 7. Copyright statement/licence for publication (see 8. Data sharing statement (see 9. Funding statement and statement of the independence of researchers from funders (see Patient involvement statement (see Please ensure the paper complies with The BMJ s style, as detailed below: a. Title: this should include the study design eg "systematic review and meta-analysis. b. Abstract: Please include a structured abstract with key summary statistics, as explained below (also see For every clinical trial - and for any other registered study- the last line of the abstract must list the study registration number and the name of the register. c. Introduction: This should cover no more than three paragraphs, focusing on the research question and your reasons

6 for asking it now. d. Methods: For an intervention study the manuscript should include enough information about the intervention(s) and comparator(s) (even if this was usual care) for reviewers and readers to understand fully what happened in the study. To enable readers to replicate your work or implement the interventions in their own practice please also provide (uploaded as one or more supplemental files, including video and audio files where appropriate) any relevant detailed descriptions and materials. Alternatively, please provide in the manuscript urls to openly accessible websites where these materials can be found. e. Results: Please report statistical aspects of the study in line with the Statistical Analyses and Methods in the Published Literature (SAMPL) guidelines Please include in the results section of your structured abstract (and, of course, in the article's results section) the following terms, as appropriate: i. For a clinical trial: Absolute event rates among experimental and control groups; RRR (relative risk reduction); NNT or NNH (number needed to treat or harm) and its 95% confidence interval (or, if the trial is of a public health intervention, number helped per 1000 or 100,000.) ii. For a cohort study: Absolute event rates over time (eg 10 years) among exposed and non-exposed groups; RRR (relative risk reduction.) iii. For a case control study:or (odds ratio) for strength of association between exposure and outcome. iv. For a study of a diagnostic test: Sensitivity and specificity; PPV and NPV (positive and negative predictive values.) v. For a systematic review and/or meta-analysis: Point estimates and confidence intervals for the main results; one or more references for the statistical package(s) used to analyse the data, eg RevMan for a systematic review. There is no need to provide a formal reference for a very widely used package that will be very familiar to general readers eg STATA, but please say in the text which version you used. For articles that include explicit statements of the quality of evidence and strength of recommendations, we prefer reporting using the GRADE system. f. Discussion: To minimise the risk of careful explanation giving way to polemic, please write the discussion section of your paper in a structured way. Please follow this structure: i) statement of principal findings of the study; ii) strengths and weaknesses of the study; iii) strengths and weaknesses in relation to other studies, discussing important differences in results; iv) what your study adds (whenever possible please discuss your study in the light of relevant systematic reviews and meta-analyses); v) meaning of the study, including possible explanations and implications for clinicians and policymakers and other researchers; vi) how your study could promote better decisions; vi) unanswered questions and future research g. Footnotes and statements Online and print publication: All original research in The BMJ is published with open access. Our open access policy is detailed here: The full text online version of your article, if accepted after revision, will be the indexed citable version (full details are at The print and ipad BMJ will carry an abridged version of your article. This abridged version of the article is essentially an evidence abstract called BMJ pico, which we would like you to write using the template downloadable at Publication of research on bmj.com is definitive and is not simply interim "epublication ahead of print", so if you do not wish to abridge your article using BMJ pico, you will be able to opt for online only publication. Please let us know if you would prefer this option. If your article is accepted we will invite you to submit a video abstract, lasting no longer than 4 minutes, and based on the information in your paper s BMJ pico evidence abstract. The content and focus of the video must relate directly to the study that has been accepted for publication by The BMJ, and should not stray beyond the data. Date Sent: 14-Apr-2016