European Food Safety Authority (EFSA), Yann Devos, Irene Munoz Guajardo Julie Glanville and Elisabeth Waigmann

Transcription

1 TECHNICAL REPORT APPROVED: 5 April 2017 doi: /sp.efsa.2017.en-1207 Explanatory note on literature searching conducted in the context of GMO applications for (renewed) market authorisation and annual post-market environmental monitoring reports on GMOs authorised in the EU market Abstract European Food Safety Authority (EFSA), Yann Devos, Irene Munoz Guajardo Julie Glanville and Elisabeth Waigmann Guidance of the Panel on Genetically Modified Organisms (GMOs) of the European Food Safety Authority (EFSA) assists applicants in the preparation and presentation of their market registration applications by describing elements and information/data requirements for the risk assessment and monitoring of GMOs. This explanatory note to the guidance: (1) clarifies the scope and methodology for literature searching performed in the context of applications for market authorisation of GMOs submitted under Regulation (EC) No 1829/2003 before and after the Implementing Regulation (EU) No 503/2013 entered into force; annual post-market environmental monitoring reports on GMOs authorised in the EU market; and GMO applications for the renewed market authorisation of GM food/feed authorised under Regulation (EC) No 1829/2003; and (2) provides detailed recommendations on how to conduct and report systematic/extensive literature searches, and present the results of any scoping reviews. European Food Safety Authority, 2017 Key words: databases, information sources, literature searching, studies, review question, scoping review, systematic/extensive literature search Requestor: EFSA Question number: EFSA-Q Correspondence: gmo@efsa.europa.eu EFSA Supporting publication 2017:EN-1207

2 Acknowledgements: EFSA wishes to thank the following for the support provided to this scientific output: Elisa Aiassa, Fernando Álvarez, Hermann Broll, Giacomo De Sanctis, Antonio Fernandez Dumont, Andrea Gennaro, Anna Lanzoni, Nikoletta Papadopoulou, Konstantinos Paraskevopoulos and Matthew Ramon, and experts of the GMO Panel standing Working Groups on Molecular Characterisation, Food/Feed, and Environmental Risk Assessment for inspiring discussions that helped to develop this explanatory note to the guidance. Suggested citation: EFSA (European Food Safety Authority), Devos Y, Guajardo IM, Glanville J and Waigmann E, Explanatory note on literature searching conducted in the context of GMO applications for (renewed) market authorisation and annual post-market environmental monitoring reports on GMOs authorised in the EU market. EFSA supporting publications 2017:EN pp. doi: /sp.efsa.2017.en-1207 ISSN: European Food Safety Authority, 2017 Reproduction is authorised provided the source is acknowledged. 2 EFSA Supporting publication 2017:EN-1207

3 Summary This explanatory note to the guidance clarifies the scope and methodology for literature searching performed in the context of applications for market authorisation of genetically modified organisms (GMOs) submitted under Regulation (EC) No 1829/2003 before and after the Implementing Regulation (EU) No 503/2013 entered into force; annual post-market environmental monitoring (PMEM) reports on GMOs authorised in the EU market; and GMO applications for the renewed market authorisation of GM food/feed authorised under Regulation (EC) No 1829/2003. It also gives recommendations on how to conduct and report systematic/extensive literature searches, and to present the results of any scoping reviews, thereby complementing previous guidance by the European Food Safety Authority (EFSA) and its GMO Panel. Specific recommendations are given for: (1) identifying review questions and clarifying their purpose; (2) searching for/identifying relevant studies; (3) selecting studies; (4) extracting high level data from the relevant studies, where appropriate; and (5) summarising and reporting the data, and considering the implications of the findings. This guidance is intended to provide a more rigorous and standardised approach to literature searching in the context of (renewal) GMO applications and annual PMEM reports. It aims to assist applicants to perform more consistent and sensitive literature searching, and ensure that as many relevant studies as possible are retrieved to minimise biases such as publication bias. 3 EFSA Supporting publication 2017:EN-1207

4 Table of contents Abstract... 1 Summary Introduction Background and Terms of Reference as provided by the requestor Scope and intended users of the explanatory note Transition period Data and Methodologies Data Methodologies Literature searching in the context of GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 after the Implementing Regulation (EU) No 503/2013 entered into force Identifying review questions and clarifying their purpose Searching for/identifying relevant studies Constructing the search strategy Approaches to develop searches Search terms Free-text searching functions Search strings Key elements of review questions Language Time period Reference study searches Appraisal of the search strategy Identifying sources of scientific literature Electronic bibliographic databases Internet searches Manual searches Appraisal of information sources used Selecting studies Process Quality assurance Eligibility/inclusion criteria to establish relevance Classification of studies Extracting of high level data from the relevant studies Summarising and reporting the data, and considering the implications of the findings Summarising and reporting the data Search methods and outcomes Results of the study selection process Narrative synthesis/summary of relevant studies Considering the implications of the findings Value of undertaking systematic review Implications for risk assessment Literature searching in the context of GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 before the Implementing Regulation (EU) No 503/2013 entered into force; annual PMEM reports on GMOs authorised in the EU market; and GMO applications for the renewed market authorisation of GM food/feed authorised under Regulation (EC) No 1829/ References Glossary EFSA Supporting publication 2017:EN-1207

5 Appendix A Overview of the main categories of information/data requirements according to current relevant GMO Panel guidance documents, EFSA explanatory notes and the Implementing Regulation (EU) No 503/2013, for which scientific literature should be searched30 Appendix B Search strategy examples Appendix C Examples of web-based databases that can contain relevant information supporting the risk assessment of GMOs Appendix D Requirements for undertaking scoping reviews applicable to systematic/extensive literature searching EFSA Supporting publication 2017:EN-1207

6 1. Introduction Genetically modified organisms (GMOs) are subject to a risk assessment and regulatory approval before entering the market in the European Union (EU). In this process, the role of the European Food Safety Authority (EFSA) is to independently assess and provide scientific advice to risk managers on any possible risk that the use of GMOs may pose to human and animal health and the environment. Decisions on the level of acceptable risk are taken by risk managers who weigh policy options to accept, minimise or reduce characterised risks. Risk managers may also request that monitoring be conducted after the GMO is placed on the market. Risk assessment is performed in accordance with the principles and information/data requirements outlined in guidance documents developed by EFSA and its GMO Panel, and relevant legislation such as Regulation (EC) No 1829/2003 and the Implementing Regulation (EU) No 503/2013 on GM food/feed, and Directive 2001/18/EC on the deliberate releases of GMOs into the environment. Robust risk assessments begin with an explicit problem formulation where plausible and relevant exposure scenarios and the potential adverse effects from those exposures are identified. Risk is then characterised by testing specific hypotheses about the likelihood and severity of adverse effects. To ensure that all relevant information is considered during the risk assessment, literature searching is typically performed by applicants in the context of GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 (referred to hereafter as GMO applications), annual post-market environmental monitoring (PMEM) reports on GMOs authorised in the EU market, and GMO applications for the renewed market authorisation of GM food/feed authorised under Regulation (EC) No 1829/2003. Literature searching is undertaken to find as much relevant studies as possible in support of the risk assessment. From the experience gained during the evaluation of literature searches reported in the context of (renewal) GMO applications and annual PMEM reports, EFSA noted that applicants have undertaken such searches to various degrees of rigour. To provide a more rigorous and standardised approach to literature searching, EFSA identified the need to develop more prescriptive guidance on how to conduct and report systematic/extensive literature searches and present the results of any scoping reviews. This guidance is intended to assist applicants to perform more consistent and sensitive literature searching, and ensure that as many relevant studies as possible are retrieved to minimise biases such as publication bias Background and Terms of Reference as provided by the requestor The purpose of this assignment is to: (1) clarify the scope and methodology for literature searching performed in the context of GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 before and after the Implementing Regulation (EU) No 503/2013 entered into force, annual PMEM reports on GMOs authorised in the EU market, and GMO applications for the renewed market authorisation of GM food/feed authorised under Regulation (EC) No 1829/2003; and (2) provide detailed recommendations on how to conduct and report systematic/extensive literature searches and present the results of any scoping reviews Scope and intended users of the explanatory note The recommendations given in this explanatory note are directed to applicants that conduct literature searching in the context of their: GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 before and after the Implementing Regulation (EU) No 503/2013 entered into force on 8 December 2013; Annual PMEM reports on GMOs authorised in the EU market; GMO applications for the renewed market authorisation of genetically modified (GM) food/feed authorised under Regulation (EC) No 1829/2003. Intended users also include the Competent Authorities of EU Member States and EFSA which are responsible for evaluating GMO applications and annual PMEM reports, and for drawing conclusions on potential risks that GMOs may pose to human and animal health and the environment in the EU. 6 EFSA Supporting publication 2017:EN-1207

7 This explanatory note may be revised when experience is gained in its application and in view of any amendments to the Implementing Regulation (EU) No 503/ Transition period The requirements outlined in this explanatory note are fully applicable to all (renewal) GMO applications and annual PMEM reports submitted at least six months after the publication date of this note. 2. Data and Methodologies 2.1. Data In developing this explanatory note, EFSA took into account the information/data requirements outlined in relevant legislation such as Regulation (EC) No 1829/2003 and the Implementing Regulation (EU) No 503/2013 on GM food/feed, and Directive 2001/18/EC on the deliberate releases of GMOs into the environment, and guidance documents developed by EFSA and/or its GMO Panel (EFSA, 2010a,b; 2011a,b,c,d,e; 2013; 2015a,b,c). Information from published scientific literature was also considered when relevant to: (1) clarify the scope and methodology for literature searching; and (2) provide detailed recommendations on how to conduct and report systematic/extensive literature searches and present the results of any scoping reviews Methodologies EFSA took into account the principles for literature searching described in its guidance on the application of systematic review methodology to food/feed safety assessments to support decisionmaking (EFSA, 2010a), and relevant scientific publications (e.g., EFSA, 2011a; Glanville et al., 2014). This explanatory note was prepared with the assistance of EFSA s GMO Panel. 3. Literature searching in the context of GMO applications for market authorisation submitted under Regulation (EC) No 1829/2003 after the Implementing Regulation (EU) No 503/2013 entered into force According to Article 6(1) of the Implementing Regulation (EU) No 503/2013, the application shall include a systematic review of studies published in the scientific literature and studies performed by the applicant within the period of ten years prior to the date of submission of the dossier on the potential effects on human and animal health of the GM food and feed covered by the application. This mandatory requirement applies to all GMO applications submitted for regulatory review under Regulation (EC) No 1829/2003 after 8 December Systematic literature review is a potentially powerful tool to support risk assessments; it provides approaches for researchers to identify, collect, appraise and synthesise available evidence on specific research questions in a methodologically rigorous, standardised and reproducible manner. Further, the systematic review approach helps to minimise bias and random error in the preparation of reviews, and ensures transparency (EFSA, 2010a; Aiassa et al., 2015; Kohl et al., 2015; Haddaway and Bilotta, 2016). Applying systematic literature review methods can be resource intensive and time consuming compared to other approaches to conducting reviews. Moreover, systematic literature reviews are best utilised to provide answers to focused, well-defined review questions (EFSA, 2010a; Aiassa et al., 2015). However, it is possible that the evidence base for the specific GMO under consideration is limited or non-existent, or that the scientific uncertainty around a topic is low. Under these circumstances it may not always be useful or necessary to perform the complete systematic literature review process (Kohl et al., 2015). Consequently, a GMO application submitted for regulatory review would not comply with all information/data requirements outlined in the Implementing Regulation (EU) No 503/2013. In these situations, the applicant shall submit reasoned justification for the derogation according to Article 5(2) of the Implementing Regulation (EU) No 503/2013, explaining 7 EFSA Supporting publication 2017:EN-1207

8 why it may not be useful to perform a full or rapid systematic literature review (Harker and Kleijnen, 2012; Kohl et al., 2015; Schünemann and Moja, 2015). To substantiate decisions about the value of conducting full or rapid systematic literature reviews for the specific GMO under consideration, applicants should scope the scientific literature. Scoping is a useful and increasingly popular approach to assessing the literature and it can be used to collect and organise important background information and develop a picture of the existing evidence base of a broad field, and thus support decisions about the value of conducting more focused systematic literature reviews (Arksey and O Malley, 2005; EFSA, 2010a; Levac et al., 2010; Armstrong et al., 2011; Daudt et al., 2013; Colquhoun et al., 2014; Peters et al., 2015; Moher et al., 2016; Tricco et al., 2016). Thoroughly defining the scope of review questions is a crucial step in the process of systematic literature review (EFSA, 2010a). This requires an understanding of existing literature, including gaps and uncertainties, a clarification of any definitions related to the review questions and an understanding of the way in which these are conceptualised within existing literature. Scoping reviews have been undertaken to various degrees of rigour, from non-systematic to systematic, with the approach adopted depending on the resources available (EFSA, 2010a). The lack of consensus on the scoping review methodology has led some authors to develop a methodological framework, consisting of six stages, for the conduct of scoping reviews (Arksey and O Malley, 2005; Levac et al., 2010; Daudt et al., 2013; Colquhoun et al., 2014). Key elements of this framework that are applicable to EFSA s work pertaining to the risk assessment of GMOs include: Identifying review questions and clarifying their purpose (Section 3.1); Searching for/identifying relevant studies (Section 3.2); Selecting studies (Section 3.3); Extracting high level data from the relevant studies (Section 3.4); Summarising and reporting the data, and considering the implications of the findings (Section 3.5). Scoping reviews require a protocol which a priori defines: (1) the review questions; (2) the search strategy; and (3) the explicit methods for selecting, categorising and summarising the evidence base. The protocol should comply with the relevant principles and criteria outlined in EFSA (2010a) and recommendations given in the following sections of this document Identifying review questions and clarifying their purpose Applicants should specify the problem that the scoping review is addressing in the form of clear, unambiguous and structured questions before the review begins. Given that the focus is on summarising the breadth and type of evidence, review questions should be broad in nature, but with a clearly articulated scope of inquiry. Applicants should clarify the purpose for undertaking the scoping review, and link it to the review questions. This will provide a clear rationale for undertaking the review, and support decision-making about study selection (relevance) and data extraction in the subsequent stages of the review. Because literature searching covered by this explanatory note is conducted in the context of (renewal) GMO applications and annual PMEM reports, review questions should be formulated as questions that focus on potential adverse effects on human and animal health and the environment of the GMO covered by the application. Review questions should be broken down into their key elements, in order to guide the development of search terms, structure the search, and inform the selection of relevant studies (EFSA, 2010a). Applicants should identify and specify the key elements to formulate relevant and focused review questions. These elements are the components of the review question that identify what information must be provided in a study to evaluate the intervention under consideration, and hence answer the question. Depending on the type of questions, these elements might include among others Population(s) [P], Intervention(s) [I] or Exposure [E], Comparator(s) [C], and Outcome(s) [O], and each key element used must be specified in detail (EFSA, 2010a; Aiassa et al., 2015). 8 EFSA Supporting publication 2017:EN-1207

9 Types of questions suitable for scoping reviews (see EFSA, 2010a and Aiassa et al., 2015 for concrete examples in the food/feed safety area) are: - Intervention or exposure questions which are likely to have a PICO or PECO structure (the key elements being Population, Intervention/Exposure, Comparator, Outcomes). This structure may be used when the review is evaluating the effects of an intervention or exposure. Review questions with PICO or PECO structure relevant for GMO risk assessment can be exemplified as follows: Does either the GMO 1 and derived food/feed products, or the intended trait(s), have adverse effects on human and animal health and the environment? [P = human and animal health and the environment; I/E = GMO, derived food/feed products, and/or the intended trait(s) (e.g., the newly expressed protein(s)); C = conventional counterpart, if applicable; O = adverse effects]. The PICO or PECO structure can also be expanded to include the key elements Timing and Setting in the review question (i.e., PICOTS or PECOTS ; Rooney et al., 2014); - Test accuracy questions with a PIT structure (the key elements being Population, Index test, Target condition). These can be used for reviews of test accuracy. Review questions with a PIT structure relevant for GMO risk assessment can be exemplified as follows: What is the detection limit of an index test to quantify the concentration of the newly expressed protein(s) in edible GM plant parts/tissues? [P = edible GM plant parts/tissues; I = index test; T = concentration of the newly expressed protein(s)]; - Questions on population parameters with the PO structure (the key elements being Population, Outcome). This structure may be used for reviews which aim to answer descriptive questions such as questions about prevalence, occurrence, consumption and incidence. Review questions with the PO structure relevant for GMO risk assessment can be exemplified as follows: How much of the food commodity derived from the GM plant is consumed by humans? [P = humans; O = quantity of food commodity consumed]; - Sometimes, a question will not fit into PECO, PIT or PO structures. In such a case, applicants will need to carefully identify the key elements of the question, and propose a new structure. The acronym for the question would be explained along the same lines as PECO, PIT or PO. For example, the question: Is the frequency of Cry1Ab-resistance alleles below 3% in European and Mediterranean corn borer populations from EU areas of high adoption rate (> 60%) of maize MON810? has the structure PES. This could be defined as: Population = European and Mediterranean corn borer populations; Exposure = GMO and/or the intended trait (e.g., the newly expressed protein; and Setting = geographic EU areas of high adoption of maize MON810. Review questions can also be represented by the categories of information/data requirements outlined in relevant GMO Panel guidance documents, EFSA explanatory notes and the Implementing Regulation (EU) No 503/2013 (see Appendix A for an overview). Studies relevant to the (renewal) GMO applications and annual PMEM reports are those that inform one or more information/data requirement(s) for the GMO under consideration (including the intended trait(s), and derived food/feed products) (see Section 3.3.3). As part of the review protocol, applicants should unambiguously specify the eligibility/inclusion criteria for assessing the relevance of studies for inclusion in the scoping review (see Section 3.3.3). Carefully defined eligibility/inclusion criteria inform various steps of the review methodology. They will influence the development of the search strategy by specifying the types of populations, types of intervention/exposure and, in some cases, the types of outcomes to address in the context of the PICO/PECO questions (see Section 3.2.1), and guide decisions on the information sources to search in the scoping review (see Section 3.2.2). Developing eligibility/inclusion criteria is an iterative process that starts with a clear analysis of the key elements of the review questions, and the different components that characterise the information/data requirements, to set the characteristics of the relevant studies. A preliminary search of the literature may be useful to test and refine the relevance criteria on a subset of studies or fulltext documents, to assess their applicability. 1 In the case of stacked events, all possible subcombinations should be covered, independently of their origin 9 EFSA Supporting publication 2017:EN-1207

10 Once applicants have determined the eligibility/inclusion criteria, they should be clearly reported, using Table 1 as example. To avoid missing relevant studies, the relevance criteria should not be too restrictive. Only clearly irrelevant studies should be excluded. EFSA does not consider study reliability and language suitable exclusion criteria, and therefore does not accept their use. Table 1: Examples of eligibility/inclusion criteria to establish the relevance of retrieved studies Concepts Criteria Comment Intervention/exposure Population Outcomes Comparator Information/data requirements Plant species (a) Key elements of review questions with PICO/PECO structure The GMO, derived food/feed products, This enables the selection of studies that and/or the intended trait(s) (e.g., newly address the GMO, derived food/feed expressed protein(s)) addressed in the products, and/or the intended trait(s) study are identical or similar to those under consideration under regulatory review Human and animal health, and/or the environment (including biodiversity, ecosystem services, service providing units, and endangered species) are addressed as general protection goals Effects/impacts on human and animal health, and/or the environment are addressed If the study is a comparative study that uses plant material as test material, eligible studies must report a non-gm variety Additional concepts Data inform one or more information/data requirement(s) outlined in Appendix A for the GMO and derived food/feed products under consideration, including the intended trait(s) The same plant species as the GMO under consideration is addressed in the study From the studies that address the GMO under consideration, those that address protection goals relevant to the risk assessment of the GMO are eligible Studies that address the GMO under consideration will also need to address effects/impacts on entities of concern, and potential determinants of exposure that place these entities at risk relevant to the risk assessment of the GMO Studies that address the GMO under consideration will also include a conventional counterpart as comparator in those cases where the study reports a comparative analysis study and uses plant material as test material Studies that potentially contribute to the knowledge informing the risk assessment of the GMO under consideration, and thus the risk hypotheses addressed, taking account of both hazard and exposure, can be seen as relevant using this criterion. Studies addressing other issues such as benefits, socio-economics, ethics, crop protection, detection methods, efficacy, public perception and risk communication could be excluded using this criterion, as they are not necessarily relevant to the risk assessment of GMOs This criterion permits the exclusion of studies on GMOs that contain the same intended trait(s) as the GMO under consideration, but which are introduced in another plant species 10 EFSA Supporting publication 2017:EN-1207

11 Table 1: Continued Concepts Criteria Comment Scope of GMO application Target pests/organisms (b) Stacked events Stacked events/subcombinations Access Reporting format Reporting format The pathways and level of exposure to the GMO, derived food/feed products, and the intended trait(s) addressed in the study are relevant for the intended uses of the GMO and derived food/feed products under regulatory review Target pests/organisms addressed in the study are established in the EU The single event(s) of the stacked event is (are) addressed in the study A subcombination or subcombinations of the single events of the stacked event is/are addressed in the study, independently of its/their origin Full-text document is accessible Original/primary data are presented in the study A study should only be presented once, but if it is presented in more than one study, all of the studies should be listed and grouped From the studies that address the GMO under consideration, those that consider pathways and levels of exposure relevant to the scope of the GMO application (i.e., import and processing for food/feed uses, cultivation) are eligible This permits the exclusion of studies that address interactions between the GMO and target pests/organisms that do not occur in the EU This permits the exclusion of studies on the single events of the stacked event, as the risk assessment of GMO applications for stacked events covers only the products within the scope of the GMO application i.e., the stacked event and subcombinations of the singles involved, independently of their origin This permits the selection of studies on subcombinations of the single events of the stacked event that are within the scope of the GMO application, independently of their origin If potentially relevant full-text documents cannot be obtained, then they should be listed in a table with a description of the (unsuccessful) methods that have been used to try to obtain a copy This permits the exclusion of studies that do not present original/primary data (e.g., reviews, editorials, position papers). Reviews will only be included if they present data that are not available from a primary research study Duplicate studies should be excluded at the screening stage. Only one copy of a study is required even if it is identified in more than one database (a): This is not applicable to GMO applications for cultivation and review questions that aim at determining the safe use of the intended trait(s) of the GMO under consideration (b): This is not applicable to GMO applications for cultivation 3.2. Searching for/identifying relevant studies Literature searching for scoping reviews involves the development of a search strategy (combinations of search terms) and identification of information sources to search, in order to retrieve as many relevant studies as possible in the most unbiased way possible. Biases in the selection of studies should be minimised by using an extensive and reproducible search strategy, and by the transparent reporting of how studies are identified, selected and included in the review. Ideally, the search should be performed by an information specialist (in collaboration with the review team). The information specialist has expert knowledge of: (a) structuring searches to capture review questions; (b) the different characteristics of various bibliographic databases and database interfaces; and (c) how to adapt searches to work efficiently for different information sources. The role of the information specialist is therefore fundamental to develop appropriate search strategies, identify appropriate and relevant information sources, and guarantee the extensiveness of the information retrieved (EFSA, 2010a) EFSA Supporting publication 2017:EN-1207

12 Applicants should report in detail the aim of the search and the search method (the search strategies and information sources used), so as to allow risk assessors and risk managers to judge how much of the relevant literature is likely to have been found. EFSA recognises that there may be a trade-off between the breadth and comprehensiveness (or depth) of evidence covered if the evidence base is very large, or the applicant does not have access to some information sources. Applicants are therefore encouraged to ensure that decisions surrounding feasibility do not compromise their ability to answer review questions, or achieve the review purpose. The principles of extensive and sensitive literature searches are given below Constructing the search strategy Applicants shall develop appropriate search strategies. The search terms and their combinations should be designed to be sensitive, in order to retrieve as many relevant studies as possible related to the GMO under consideration and their potential risks to human and animal health and the environment. Applicants should describe clearly how they have constructed the search strategy, and provide a scientific rationale for choices made in terms of search terms and their combination. Developing a search strategy may take several iterations during which the strategy is drafted, tested, assessed, and then revised. The experience gained through these iterations could be used to substantiate the constructed search strategy. Search strategies should be adapted to run successfully in different information sources or databases, covering synonyms, abbreviations and spelling variants (see Section 3.2.2). Applicants will need to identify and include relevant synonyms, abbreviation and spelling variants to compensate for the fact that the information available to be searched within database records may be quite brief, and to account for the fact that the words that authors use to describe their research methods and results can vary. Any limits applied to the search strategy (for example, languages, or the exclusion of specific publication types such as commentaries, letters, notes or editorials) should be explicitly reported and justified, as these decisions may introduce biases Approaches to develop searches Two different approaches (often characterised as lumping and splitting ) can be used to develop searches for multiple questions (EFSA, 2010a, 2011a): Lumping uses a single search strategy that potentially captures all review questions or categories of information/data requirements of interest (Appendix A) in one search (e.g., by searching for the GMO alone); Splitting uses a series of search strategies designed to capture individual review questions or grouped categories of information/data requirements. An advantage of the lumping approach is that the search is likely to be highly sensitive and less time consuming to run than a series of more focused searches, and it is likely to produce fewer duplicate records. Relevant studies will need to be classified according to the review questions or categories of information/data requirements that they may inform. A disadvantage of the lumping approach is that potentially a large number of search results may be returned that would need to be assessed for relevance to each of the review questions or categories of information/data requirements. If the number of studies returned by a single search strategy is extremely large, the splitting approach might be preferred, as using separate review questions might provide more manageable numbers of studies to assess. EFSA does not recommend any specific approach for the search strategy, and applicants may choose the most helpful on a case by case basis EFSA Supporting publication 2017:EN-1207

13 Search terms To design a sensitive search strategy, applicants should employ a combination of a wide range of free-text terms 2 (searching title and abstract of studies) and subject indexing terms 3, where available (selected from the indexing language or thesaurus of electronic bibliographic databases). Free-text terms (also called text words) are used by authors in the title and abstract of their studies such as journal articles; these terms are then searchable in the title and abstract fields of database records. The challenges of free-text searching are that authors may not describe their work consistently and they may not provide all the relevant information we are seeking in the title or abstract of the study. Therefore, a highly sensitive free-text search should incorporate a wide variety of synonymous and related terms, while also taking into account different conventions in spelling and variations in the endings of terms to include, for example, singular and plural terms. Subject indexing terms (where provided) are valuable in increasing the sensitivity of a search, as they provide a way of retrieving studies that are described by authors using very different vocabulary. They can also provide information which is beyond that contained in the title and abstract (Glanville et al., 2014). Index terms can be identified using the database thesaurus (if available), or by searching for obvious relevant studies and noting the indexing terms within those records (EFSA, 2010a) Free-text searching functions Specific search functions can enhance the sensitivity 4 or precision 5 of free-text searching. These features vary across information sources; the help file for each source should therefore be consulted for further guidance. Truncation: Truncation is a searching technique used in electronic bibliographic databases in which a word ending is replaced by a symbol. Truncation enables different forms of a word to be searched for simultaneously, and will increase the number of search results found. Truncation symbols, such as the asterisk (*), a question mark (?), or a dollar sign ($), are specific to the database search interface being searched. The truncation symbol is placed at the root of the word, and the database interface will find any word beginning with that root. This reduces the number of free-text terms that must be typed. Wildcards are used to account for internal spelling variation and can be particularly useful when trying to capture differences in British and US spelling. For example, in some database interfaces, randomi?ed will identify studies containing both randomized and randomised. Wildcards are specific to the database interface being searched and might be a question mark (?), a pound/hatch symbol (#), or the asterisk (*). In some database interfaces, quotation marks (" ") would be used for multi-word terms, but in others they are not required Free-text terms (also called text words) are used by authors in the title and abstract of publication; these terms are then searchable in the title and abstract of electronic records in information sources Subject indexing terms are keywords from a standardised or controlled vocabulary, or thesaurus, which are assigned to electronic records in information sources, usually by a human indexer, to describe the content of the publication Sensitivity refers to the proportion of relevant publications retrieved by a search strategy (number of relevant publications retrieved/total number of relevant publications in existence) The proportion of relevant publications among all the publications retrieved by a search strategy (number of relevant publications retrieved/total number of publications retrieved) 13 EFSA Supporting publication 2017:EN-1207

14 Search strings Search terms should be combined using Boolean operators ( OR, AND and NOT ) and proximity operators where offered by the database search interface (Glanville et al., 2014). Boolean operators: OR operator: This operator is inclusive, and is used to capture relevant synonyms within a single key element, making the search results larger. AND operator: The AND operator is usually used to combine different elements of the review question to narrow down (focus) the search. The AND operator requires that the key elements combined are present in each of the studies retrieved. NOT operator: This operator is used to exclude studies from the search. The NOT operator should be used with great care, because it may have a larger effect than anticipated. A study may well discuss both the concept of interest and the one to be excluded and might be lost from a result set if an irrelevant term is excluded from the search explicitly using the NOT operator. Proximity operators: Proximity operators are search commands which indicate that a search term can be retrieved within a certain distance from another search term. The distance may be a certain number of words, or within a sentence or a paragraph. Some database search interfaces to information sources allow a searcher to specify that free-text terms should appear adjacent to each other. In other interfaces, it is possible to specify that the terms should appear within a specific number of words of each other. This feature is especially useful when trying to account for very variable vocabulary. For example within the Ovid search interface, bacteria$ adj3 (load$ or level$ or log or logs or logging or logged or count$) will search for bacteria or bacterial within three words of any of the terms in the set of brackets, in either direction. This search string would therefore identify a range of word groupings including bacterial load, levels of bacteria, bacteria counts, and logged numbers of bacteria. Proximity operators offer better precision than using the AND operator, as terms linked by AND may be widely separated within a record and semantically unrelated. The proximity operators available will vary by database search interface Key elements of review questions A search should be structured to reflect at least some of the key elements of the research question(s), but it is unlikely and often undesirable to use all of the key elements in the search (Glanville et al., 2014). This is because search strategies with many key elements may be too precise and lack sensitivity; a typical database record with only a title, abstract and subject indexing may not contain all four or five of the desired key element in a particular review question. Key elements that are too difficult to search for sensitively, particularly the outcomes, may be excluded from consideration for inclusion in the search strategy, because they are frequently not reported in the title or abstract of studies. Only those key elements that can be clearly defined and translated into search terms should be included. It is therefore best to start off with the smallest yielding element which is also likely to be the most specific; this is likely to be the intervention or exposure in reviews with a PICO or PECO structure. Only if the results of the search for the key element are too large to process should additional key elements be added to the strategy. When additional key elements have to be added because of the volume of results, the most likely additional key element to be added into the strategy for a PICO or PECO strategy is the population of interest. If the results of a strategy structured as intervention/exposure AND population in a PICO/PECO review are too numerous to process, then the addition of a further key element can be explored. Appendix B provides a structure for developing a search strategy that indicates how variants of the key elements need to be captured to ensure a sensitive search. For example, the intervention needs to be captured using both event terms, trade names and the intended trait(s) (e.g., newly expressed 14 EFSA Supporting publication 2017:EN-1207

15 protein(s)). We offer Appendix B as examples of how to conceptualise the searches to account for search term variations. The challenges of constructing search strategies for some complex review questions have led to the use of multi-stranded strategies. Instead of a single combination of key elements such as PI or IT, a series of key element combinations might be developed to capture the various ways records express the topics. For example, multiple different approaches could be run sequentially and then combined using the OR operator as follows for a PIT review: 1 Index test AND condition 2 Condition AND reference standard 3 Index test AND diagnostic test accuracy terms (search filters) 4 1 OR 2 OR Language If the searches are performed in databases/interfaces other than English, then applicants should adapt the search strategy to reflect those languages Time period In line with Article 6(1) of the Implementing Regulation (EU) No 503/2013, the search strategy for GMO applications must be capable of capturing studies published during the ten years prior to the date of submission of the GMO application. Although this is not mandatory, studies may also be included from more than ten years prior to GMO application submission, provided that studies are identified and selected in compliance with EFSA (2010a), and appropriate justification is provided Reference study searches Applicants shall use a subset of representative studies, if available, complying with the eligibility/inclusion criteria, in order to test, fine-tune and validate the search strategy as part of the protocol development. This testing should enable applicants to check if the strategy suggested for the scoping review is suitable to retrieve the already known literature on the topic. The retrieval of all reference studies within the results in the queried information sources suggests that the search strategy is performing at least adequately, and provides a means to validate the searches. The compilation of representative reference studies, covering all relevant categories of information/data requirements, can be made out of the list of references of recent reviews on relevant topics and methodological studies, as well as being informed by the applicants experience in the subject Appraisal of the search strategy Literature searches can be assessed for quality using the EFSA critical appraisal tool (CAT) for systematic/extensive literature searches (Appendix D of EFSA, 2015b). The CAT focuses on two aspects of literature searching: (a) the search strategy; and (b) the information sources used. Applicants may wish to assess their strategy using the CAT Identifying sources of scientific literature Identifying research evidence requires choosing and searching a range of information sources. The choice of sources will depend on the review topic. There is no agreed acceptable number of information sources which should be searched to inform a scoping review; this is dependent on the focus of the review questions and the resources available to the review team (Glanville et al., 2014). Searching multiple information sources increases the likelihood of retrieving unique studies. It also reduces the impact of variability of indexing within information sources; a study that is not retrieved by a search strategy in one information source may be retrieved by a similar search strategy used in another information source (Glanville et al., 2014). There may be a number of different sources that will yield relevant studies (e.g., electronic bibliographic databases recording studies such as journal articles, books/monographs, dissertation 15 EFSA Supporting publication 2017:EN-1207

16 theses, conference proceedings, reports and other grey literature). It is therefore important that applicants consider the topic of their review questions when identifying information sources to search, and their combination. The selection of information sources should aim to enable identification of both published and unpublished research (unpublished research includes documents that are often described as grey literature), in order to reduce the impact of publication bias. Where possible, the selection of information sources should also facilitate retrieval of research published in languages other than English, as well as on-going and recently completed research (Glanville et al., 2014). Key criteria to guide the selection of information sources are shown in Table 7 of Glanville et al. (2014), and applicants may wish to use this as an information source selection checklist. In addition, relative recall methods can be used to prospectively determine the best combination of information sources for the review topic (for further information, see Glanville et al., 2014). Searching various information sources is likely to result in duplicate studies which can be removed by de-duplication. We note that different reports of the same study may also be identified. In the latter case, these studies should be grouped and treated as a single study to avoid double counting Electronic bibliographic databases The objective, when selecting information sources to search, is to identify those information sources likely to yield relevant studies (Glanville et al., 2014). Electronic bibliographic databases are the most efficient way to identity an initial set of relevant studies published as journal articles. They are usually designed to facilitate effective information retrieval with information presented in structured ways and often with the addition of indexing. They can be searched using both free-text terms and indexing terms, where available. Most electronic bibliographic databases include abstracts for the majority of recent studies, and may include links to the full-text of the study, where available. Applicants should therefore make reasonable efforts to locate relevant sources of scientific evidence, focusing on major relevant databases. Using an appropriate range of information sources will minimise the impact of publication bias, and help to address limitations in research reporting and indexing. The number of databases to consider may differ between topics, but a minimum of at least two multi-disciplinary/large databases (e.g., Web of Science Core Collection, Scopus, CAB Abstracts, Medline) allows some consideration of complementarity and redundancy (O Connor et al., 2012). Applicants shall also consider searching more specialist/subject-specific databases (e.g., Agricola). Databases that index scientific evidence relevant to EFSA s food/feed safety assessments can be identified from Glanville et al. (2014). Applicants should describe the information sources they have searched, and the reasons for their selection in terms of the coverage of the literature of interest should be specified Internet searches Applicants should complement the searches of electronic bibliographic databases with internet searches to identify relevant literature through additional approaches. The identification of potentially relevant studies outside of electronic bibliographic databases may yield additional evidence that can mitigate effects of publication bias (e.g., Haddaway et al., 2015, 2016). EFSA acknowledges that retrieving and reviewing grey literature can be resource intensive, with limited returns. Applicant should search internet pages of relevant key organisations (such as regulatory agencies and risk assessment bodies involved in the risk assessment of GMOs), which deliver risk assessment guidelines and other risk assessment documents, ranging from scientific opinions/reports concerning the commercial release of GMOs to companion documents such as biology documents on major crops 6 [e.g., US Environmental Protection Agency (EPA); US Department of Agriculture (USDA); US Food and Drug Administration (FDA); Canadian Food Inspection Agency (CFIA); Health Canada; Environment 6 CFIA (Canadian Food Inspection Agency): /biology-documents/eng/ / ; OECD (Organisation for Economic Co-operation and Development): ybiologyofcrops.htm; and OGTR (Office of the Gene Technology Regulator): EFSA Supporting publication 2017:EN-1207