Author's response to reviews Title: Ethics review of studies during public health emergencies - The experience of the WHO Ethics Review Committee during the Ebola Virus Disease epidemic Authors: Emilie Alirol (Emilie.Alirol@hcuge.ch) Annette Kuesel (kuesela@who.int) Maria Guraiib (guraiibm@who.int) Vania de la Fuente-Núñez (delafuentenunezv@who.int) Abha Saxena (saxenaa@who.int) Melba Gomes (gomesm@who.int) Version: 1 Date: 16 Apr 2017 Author s response to reviews: Reviewer Comments Phaikyeong Cheah (Reviewer 1): This is an important paper that describes lessons learnt from the recent ebola outbreak. Minor comments below. Response: Thank you. 1. WHO ERC - please describe the WHO ERC and its composition (as well the EVD subcommittee). Are there any independent or lay members, how many people are in the committee? Is it a joint scientific and ethics committee? Response: We have expanded the section 'Application of WHO-ERC Rules of Procedure' to include: - WHO-ERC composition - Requirement for submissions to include assessment and approval of independent scientific reviewers - More details on the disciplines represented on the WHO-ERC EVD subcommittee
- use of CIOMS guidelines - Website with details on WHO-ERC and review process. 2. Last sentence in the "Background" - "This paper describes challenges encountered and reflects lessons " does not reflect what is in the headings/ body of the paper. Response: We have modified the sentence as follows: This paper describes our experience and reflects lessons for future public health emergencies (PHE). and changed the heading 'Conclusions' to 'Recommendations for future outbreaks/phes' 3. Pg 4, "beneficence: risks, benefits and benefit-risks ratio a) - suggest including some references e.g. how to balance risks and benefits, what are the types of risks and benefits (indirect vs direct, benefits to individual, family, community, immediate vs future) Response: We have added the following references: CIOMS guideline Rid A, Wendler D. Risk-benefit assessment in medical research--critical review and open questions. Law, Probab. Risk. 2010;9:151 77. Kimmelman J, Henderson VC. Assessing Risk/Benefit for Trials using Preclinical Evidence: A Proposal. J. Med. Ethics. 2016;42:50 3. b. - please be consistent: benefit-risk ratio vs risk-benefit ratio Thank you for pointing out the inconsistencies. We have now used benefit-risk throughout. 4. page 7, end of 3rd paragraph. Why didn't the ERC insist on protocol amendments to include pregnant women/children? Presumably it was because it did not want to delay the approval process. Did the local ECs ask for inclusion of children and pregnant women? Response: The reasons were provided in the original manuscript: Given the time pressure for study initiation and mindful that legal issues might be driving exclusion of these groups... We have expanded this sentence to be more explicit about our reasons for not insisting on inclusion of pregnant women/children: Given the time pressure for study initiation and mindful that legal issues might be driving exclusion of these groups, the trade-off was between 'delayed initiation with potential benefit for
all' versus 'immediate initiation with potential benefit for the majority' and WHO-ERC did not insist on protocol amendments when data-based justifications were not provided. The approvals from the local EC did not reflect their discussions and since protocol amendments including pregnant women/children were not submitted, we conclude that local ECs did not insist on inclusion of children and pregnant women either. 5. Page 7, "Informed voluntary" consent - what guidelines/reference did the ERC use? Did the ERC have its own guidelines for what constitutes "informed voluntary"? (see also the definition of "valid consent") Response: The ERC uses the CIOMS guidelines. We added this information in the section on Application of WHO-Rules of Procedure. 6. "Conclusions" - these appear to be reflections/lessons learnt/recommendations instead of conclusions Response: We agree. The title of the section has been changed to: Recommendations for future outbreaks/phes 7. page 9, "framework for scientific " - can you suggest a legal framework for the Joint Ethics Review Committee? Response: Our intent was not to imply a legal framework but to propose a collaborative approach. We have modified the heading to Scientific, data and safety monitoring and ethical review. In response to this comment and a comment from Reviewer 3, we have added the following sentence: A J-ERC might face practical and legal issues, but the advantages should motivate efforts to overcome these: We have also provided a reference to the process involved in harmonising Nordic ethical reviews of clinical trials: Nordic Trial Alliance Working Group on Ethics. Report on the Ethical Review Process for Clinical Trials in the Nordic Countries. 8. The paper ends rather abruptly. Suggest adding some concluding statements Response: We agree. We have renamed the previous 'Conclusions' section to ' Recommendations for future outbreaks/phes' and added a new concluding paragraph with the title Conclusions : Despite time and psychological pressure, we did not identify shortcuts to fulfilling our mandate conscientiously; the moral and ethical imperative was to examine the submissions as thoroughly and with the same type of considerations the WHO-ERC applies to studies outside PHE. Future outbreaks/phes will differ from the EVD outbreak in epidemiology, morbidity and mortality,
treatment options, health care system capacity and psycho-socio-economic context which affects the ethical acceptability of protocols. Our experience suggests that advance agreements on the principles governing bio-specimen and data use, prior consultations on inclusion/exclusion of pregnant women and children, and exploration of joint scientific, data safety monitoring and ethical reviews early in the next PHE could accelerate finalization of study plans and ethics approval; in other respects there is no substitute for good research protocols and thorough EC review, even under pressure. Bridget Haire (Reviewer 2): This article provides very valuable information about the processes of ethical review during the West African Ebola outbreak, and makes important recommendations about how these processes could be further streamlined in future health emergencies. Response: Thank you. 1. The abstract does not mention the importance of collaboration with affected communities (both partnerships with local researchers and consultations at grassroots levels), though this is significant in the main text (p 6, pp 7-8). Including this in the abstract would be valuable. Response: This was intended to be reflected in 'collaborative partnerships' and 'community engagement'. We have modified the sentence to be more explicit: 'collaborative partnerships including international and local researchers and communities,' (we have shortened the abstract in other areas to remain within the 350 word limit). 2. Some awkward sentences need revision, such as the first sentence of the introduction (p 3); Response: Modified to: The Ebola Virus Disease (EVD) epidemic in West Africa began in December 2013 and was declared a public health emergency of international concern on 8 August 2014. On 29 March 2016, the World Health Organization (WHO) declared that it no longer constituted such an emergency. P4 under subhead 'review times', lines 22-24; Response: Modified to: We minimised time from protocol review to approval by discussion of issues with WHO responsible officers and researchers. p6 lines 14-15 (was this intended to mean volunteers in Ebola-affected communities, perhaps?);
Response: P6, lines 14-15: The original manuscript specifies that this paragraph refers to Phase I/II vaccine trials in countries not affected by the epidemic. We have further clarified this by modifying the text as follows: The Phase I/II vaccine trials were dose-ranging placebo controlled immunogenicity and safety studies in countries not affected by the epidemic (Switzerland, Kenya, Mali, The Gambia, Gabon), with participants randomized Studies in Switzerland expected some volunteers from organizations which deployed staff to affected countries. WHO-ERC agreed that these studies could randomize volunteers who might be deployed only to vaccine arms to maximize potential for direct benefit. p8 lines 1-3. Response: P 8, lines 1-3: We have changed the sentence to: Consequently, WHO-ERC considered involvement of local scientists familiar with culture, attitudes, language and socio-economic context even more important than for other studies and indispensable despite the strain of the outbreak on local resources. 3. There are a couple of lists that throw together 2 positive aspects with a 3rd negative aspect, which is jarring to read, see p4 lines 50-58 - these lists would be easily revised to make all element on the list either positive or negative.. Response: Many thanks for pointing this out. We have changed the sentences Original manuscript p4 lines 50-58 to: Experimental interventions were evaluated on whether they were likely to have benefit (improve the chances of survival or reduce the probability of infection) or increase risk (result in serious temporary or irreversible adverse reactions, including death).. We have changed original page 5, lines 42-45 to: c) Uncertain benefit-risk ratio of investigational interventions and 'standard-of-care'; the latter differed between ETC and evolved over time, impacting assessment of the relative benefit-risk ratio of the interventions; 4. p 1 lines 6-8 - I suggest mentioning observational intervention research that occurred during past outbreaks, such as interventions that used survivor blood. Response: We have added 2 new references. Bwaka MA, Bonnet MJ, Calain P, Colebunders R, De Roo A, Guimard Y, et al. Ebola hemorrhagic fever in Kikwit, Democratic Republic of the Congo: clinical observations in 103 patients. J. Infect. Dis. 1999;179 Suppl:S1 7.
Mupapa K, Massamba M, Kibadi K, Kuvula K, Bwaka MA, Kipasa M, et al. Treatment of Ebola Hemorrhagic Fever with Blood Transfusions from Convalescent Patients. J. Infect. Dis. 1999;179:S18 23. 5. The section on scientific validity and trial design could be expanded. For example, some exploration of the term 'scientifically valid' is warranted, as it appears that RCT advocates suggest this is the only valid form of research. It seems that the WHO ERC reviewers did not accept this (incorrect) view, and it would be good to see this articulated for posterity. Response: As the reviewer points out, and elaborated in this section of the manuscript, views on what constitutes a 'scientifically valid' study design differed significantly for the EVD trials. As now specified in the section describing WHO-ERC procedures (based on request for clarification on whether the WHO-ERC is an Ethics Committee or a joint EC and scientific committee), all protocols submitted to WHO ERC have to have independent scientific review and approval. We have repeated this also now in this section. The point we are making here is that WHO-ERC considered the proposed study designs - all considered valid by the scientific review - from a number of different perspectives that impacted the validity of the justification of the proposed design from an ethical perspective (see list a-g in that section). We feel that the statement in the manuscript: WHO-ERC considered these designs appropriate for generating valid scientific knowledge without withholding the intervention from those who could potentially benefit.' Expresses WHO-ERCs assessment for posterity. 6. p 5 item c) - the issue about varying standards of care in different ETC and how these would impact on RCT designs is not clearly articulated in this point. As written, it is hard to understand. Response: We have modified the sentence as follows: Uncertain benefit-risk ratio of investigational interventions and 'standard-of-care'; the latter differed between ETC and evolved over time, impacting assessment of the relative benefit-risk ratio of the interventions; 7. P 6 - I think that in paragraph 2, last line, the authors are talking about study volunteers in Ebola-affected areas, not study volunteers generally. This needs to be clarified. Response: Please see our response to point 2 above. We have modified the text to clarify that these were studies outside the Ebola-affected areas: The Phase I/II vaccine trials were dose-ranging placebo controlled, immunogenicity and safety studies in countries not affected by the epidemic (Switzerland, Kenya, Mali, The Gambia, Gabon), with participants randomized between different vaccine doses and placebo. Studies in Switzerland expected some volunteers from organizations which deployed staff to affected
countries. WHO-ERC agreed that these studies could randomize volunteers who might be deployed only to vaccine arms to maximize potential for direct benefit. 8. p 6 paragraph 3, the explanation of the 'ring' trial does not capture the cluster element in its design. This section needs a minor rewrite to ensure the design is clear. Response: We have added the information that each ring constituted a cluster:.. it randomized contacts of new EVD cases (rings, each constituting a cluster) to immediate or delayed vaccination 21 days after randomization 9. P 6 paragraph 3 item b) - not only does the 'delayed vaccination arm' enhance social value by providing a comparator, it also has a more direct benefit to participants as they are vaccinated by a promising candidate that might provide them with protection against future exposure. The sentence in the original manuscript begins by pointing out that WHO-ERC agreed to this design '. given early phase immunogenicity data suggesting a potentially favourable benefitrisk ratio. We have clarified that any immunogenic protection would apply to all vaccinated individuals (delayed or immediate) by modifying the sentence to state: The WHO-ERC agreed with the design from the perspective of distributive justice (see below), given immunogenicity data from healthy volunteer studies suggesting a potentially favourable benefit-risk ratio to all vaccinated individuals 10. P 6 item 3 Justice - exclusion of women and children - I congratulate the authors for the clarity of this section. Response: Thank you. 11. P 8-9 item 5 - the issues of consent, partnership and community engagement are of critical importance, and as I have noted above, should also be included in the abstract. Please see our response to point 1. 12. P9 Template agreements about biospecimen ownership and use need to be negotiated with communities and local researchers during non- emergency circumstances. Response: We agree. We suggest that the ongoing discussions (which involve ministries of health and researchers from north and south) could be the basis for such template agreements. However, negotiation with communities and local researchers cannot take place before the next outbreak because we do not know where the next emergency will be.
We have clarified our recommendation by including the word 'study-specific' in the relevant sentence: Template agreements emerging from international consultations, e.g. could accelerate studyspecific agreements on. 13. Might it be possible for the authors to include a section on the monitoring and ethical evaluation of the implementation of trials post-emergency? Articulating where the responsibility for monitoring of the implementation of trials during an emergency would be a very valuable addition to this article, if it is possible, and would add greatly to the sends of transparency in the elevation of what occurred during this particular outbreak. We have added the following paragraph on our considerations regarding monitoring to section 5. Respect for persons referring to our Good Clinical Practice Guidelines require sponsors to verify study implementation per protocol through monitoring. WHO-ERC reviewed monitoring plans of all prospective studies, realizing that standard monitoring approaches were not feasible for studies in ETCs. The prospective studies involved individuals at risk of, or survivors of, Ebola Virus infection. WHO-ERC reviewed monitoring reports with particular attention to implementation of measures to improve understanding of procedures, risks and test results (see above). In one instance WHO-ERC requested an additional independent monitoring visit. We feel that post-emergency evaluation is outside the scope of our paper. SEEMA SHAH (Reviewer 3): Dear authors, 1. I was glad to have the opportunity to review this interesting paper. The idea to draw lessons from the WHO's experience in reviewing protocols during the recent Ebola virus disease epidemic is inspired, and this piece has the potential to make a significant contribution to the literature and future ethics review in outbreaks of emerging infectious diseases. The authors do a nice job of drawing from their experience, identifying some significant challenges of this kind of work, laying out the major ethical issues, and citing the relevant literature. However, the paper is not structured in a way that makes its contributions as clear or well-supported as they could be. Accordingly, I would recommend that the authors modify the structure and argumentation of the piece in several ways that I will describe below. Response: Many thanks.
We appreciate the reviewer's motivation to help improve the paper, and have carefully considered the reviewer's suggestions. 2. First, the body of the paper largely focuses on substantive ethical issues (e.g., whether forgoing a placebo control was justified in trials of treatments for Ebola virus disease and for vaccines, if pregnant women and children should have been included in these trials, etc.). However, the recommendations at the end of the paper largely focus on process issues that seem to be designed to speed up review. It would be helpful if the authors separated substantive from procedural issues in the paper, and made a tighter connection between the problems they saw in each area and their recommendations. Response: We prefer not to introduce subheadings separating 'procedural' and 'substantive' for the following reason: Our primary target audience are researchers preparing protocols who would likely view anything labelled 'procedural' as 'EC members insisting on procedures when there are lives at stake'. Furthermore, the 'reference documents' for researchers (DoH or CIOMS) do not separate 'substantive' and 'procedural'. Readers who are familiar with this separation will be able to recognize which recommendation fits which category. Our recommendations are not designed to address each items in 'WHO-ERC considerations during protocol review' because they are to be applicable across outbreaks and PHEs. We realize that the choice of the heading 'conclusions' was inappropriate since it builds the expectation of a tight connection of this section with the previous one. In response to this comment, - we have changed the title of the section including the recommendations to 'Recommendations for future outbreaks/phes'. - we have expanded the introduction to this section to - explain that protocol evaluations need to take into account the specific context of a study which limits generalizable conclusions from our review - to better characterize the common elements of outbreaks or other public health emergencies that our recommendations address. - to spell out that our recommendation address issues applicable across PHE/outbreaks. Furthermore, we have (where this was not already the case), added text to each recommendations pointing out the EVD experience that motivated the recommendation. 3. With regard to the substantive issues, the recommendation for a global consultation on the inclusion of pregnant women and children in trials conducted during outbreaks seems to be the only one tied to the substantive arguments.
Response: Please see our response to item 2. None of the recommendations directly address the issues about study design, and it is not clear if the authors think that anything about the review of study design should have been done differently. Response: Section 'WHO-ERC Considerations 2 'Beneficence - Scientific validity: Study design' in the manuscript, specifies the WHO-ERC review considerations. They show that study design needs to be considered within the specific context. Consequently, no recommendations applicable to other outbreaks/phe can be derived. As noted above, we have now spelled out in the expanded initial section of 'Recommendations for future outbreaks/phes' that protocols need to be evaluated within the specific context of a study. It is possible that more streamlined review and exchange of information across different review bodies is meant to allow for more thorough scientific review and to ensure that information is shared with ethics review committees, but such an idea is not spelled out. Response: This is not spelled out because this is not what we want to say. Our motivation for proposing joint SACs and DSMB is spelled out: ' to ensure (and reassure ECs) that recommendations for one study (e.g. dose selection) are made with detailed knowledge of the results of all other relevant studies.' Information sharing with ECs is independent of separate or joint SACs or DSMBs. The authors should also acknowledge that there are important trade-offs that were made, but note that they took a defensible position by approving the trial designs presented. Response: We have modified the sentence providing the rationale/justification for the 'trade off' on exclusion of pregnant women and children to be more explicit on the trade-off section 3: Given the time pressure for study initiation and mindful that legal issues might be driving exclusion of these groups, the trade-off was between 'delayed initiation with potential benefit for all' versus 'immediate initiation with potential benefit for the majority' and the WHO-ERC did not insist on protocol amendments when data-based justifications were not provided. An additional article that would be helpful to cite on this point is: Rid A, Miller FG. Ethical Rationale for the Ebola "Ring Vaccination" Trial Design. Am J Public Health. 2016 Mar;106(3):432-5. Response: We have added the suggested Rid reference. Alternatively or in addition, they could bring in other evidence or arguments to support their recommendations. For instance, the Guinea ring vaccination trial was complicated by the fact that, in my understanding, the study team changed the primary outcome to get definitive results.
(See Henao-Restrepo AM, Camacho A, Longini IM, et al. Efficacy and effectiveness of an rvsv-vectored vaccine in preventing Ebola virus disease: final results from the Guinea ring vaccination, open-label, cluster-randomised trial (Ebola Ça Suffit!).Lancet. 2016 Dec 23. Pii: S0140-6736(16)32621-6.) Response: The reviewer's statement the study team changed the primary outcome to get definitive results is not consistent with our understanding and involvement in protocol and amendments. See also Kieny et al. 2016, Changes in the primary outcome in Ebola vaccine trial - authors reply. After the first review of interim results in June 2015, the DSMC advised that the protocol be amended to halt randomisation. An amendment to the trial design was approved by WHO-ERC. It is possible that speeding up research review could have helped start the trial earlier in the epidemic when the risk of infection was much higher, leading to more interpretable results. This would be a good motivation for the ideas the authors have for speeding up research review. Response: The start of the study was primarily driven by availability of Phase I/II study data required before large-scale human use (in the ring-vaccination trial) could be proposed on a sound scientific and ethical basis. 4. With regard to the specific recommendations, some of them are good ideas, and others are less well-supported or less clear about how they would be put into practice. As I have noted, there is also a disjunct between the body of the paper and these recommendations. Response: Please see our response to point 2. First, the argument that investigators should take time to make sure their submissions are internally consistent is a little misplaced, at least without more information. For instance, if the inconsistencies that held up review were minor or inconsequential, then they should not have held up review and the problem was with the reviewers. If the inconsistencies were about major issues of ethical importance, only then would it be important for investigators to spend valuable time fixing up the protocol instead of moving the research forward in other ways. Response: We agree with the reviewer s comment that minor or inconsequential inconsistencies should not have held up study approval, nor did they. We provided examples for the inconsistencies referred to in the sentence and the full list is included in Table 4 which is referenced. WHO-ERC did not consider any of the inconsistencies listed in the paragraph as key elements or included in Table 4 as inconsequential.
5. Second, the idea for joint ERC is a good one but subject to several practical and potentially ethical issues. It would take time to constitute such a committee from scratch, and institutions in the U.S. might be concerned about legal liability if they were to rely on this committee rather than their own IRBs (or a standard IRB/ERC). Someone would presumably have to vet the membership of this committee to ensure it meets the standards in some or all research regulations that would apply. Response: We agree that a joint ERC may be subject to several practical and legal constraints and might take time to institute and that ECs/IRBs in some countries might not want to/be able to participate, even though, if our proposal was adopted, each EC/IRB would be represented on the Joint EC. In response to this comment we have added the following sentence before the text on the advantages of a J-ERC: A J-ERC might face practical and legal issues, but the advantages should motivate efforts to overcome these: 6. Third, the idea for a joint DSMB and Scientific Advisory committee is an interesting idea, but challenges could arise if the interim data from one trial calls into question the conduct of another, and if the data are considered confidential or proprietary. These challenges might also be a problem for the idea that there should be "direct information exchange between the chairs of advisory, safety review and ethics committees." For an article on this issue, see: Shah SK, Dawson L, Dixon DO, et al. Should sponsors and DSMBs share interim results across trials? J Acquir Immune Defic Syndr. 2011 Dec 15;58(5):433-5. Response: Interim results from one trial calling in question the conduct of another trial is an excellent example for our rationale for Joint DSMBs and Joint SACs: (quote from our manuscript: 'recommendations for one study (e.g. dose selection) are made with detailed knowledge of the results of all other relevant studies'.) Members of committees discussing proprietary data have to sign confidentiality agreements no matter how many trials or interventions are discussed. In view of this comment, we have included in the section on WHO ERC that ERC members are bound by confidentiality. We thank the reviewer for the reference which we included. The article provides a good example for the benefits of the joint committees we propose. 7. Fourth, the notion that there should be more than standard support for investigators by ethics committees sounds good in theory, but it is unclear what this means in practice. More specific suggestions would be helpful here. Response: In response to this comment, we have added an example:
(e.g. discussion of revised information documents before re-submission) to the sentence the reviewer refers to. 8. Finally, the authors put a lot of weight on improving informed consent documents as a way to build trust. The available data do not suggest that many things other than extended time for discussion and test/feedback approaches can really improve understanding, and I don't know of any data that measures whether improving informed consent can build trust. See Flory & Emanuel, JAMA 2004; Nishimura et al. BMC Med Ethics 2013. I suspect the emphasis on improving informed consent documents in the context of research on Ebola virus disease during an epidemic is misplaced, and better community engagement and social mobilization is really important. If the authors really think improving informed consent could be valuable, they should cite literature in support of this claim. Response: We emphasize that lengthy and technical information documents are inappropriate even more so when participants are critically ill or fear for their lives and that in reviewing the proposed documents, WHO-ERC was mindful of the conditions under which participants were recruited. We have carefully re-read our manuscript and cannot find instances where we state or imply that improving informed consent documents is a way to build trust.