PRISMA-P: Preferred Reporting Items for Systematic Reviews and Meta- Analyses Protocols Larissa Shamseer, MSc Reporting Guidelines Research Coordinator Knowledge Synthesis Group
Overview What is a systematic review protocol? Why are protocols important? Where can protocols be accessed? Protocols & Selective reporting How can reporting guidelines help? PRISMA-P Development Next steps
What is a systematic review protocol? States intentions for planned research Description of methodological approach Defines selection criteria Provides operational definitions Document written prior to starting a systematic review (SR) stating rationale, intended purpose and content (i.e. methods)
Why are reporting protocols important? Potential to reduce bias & enhance rigour Enhance transparency and confidence in ensuing review Reduce duplication of effort and foster collaboration Assist in critical appraisal Protect/uphold integrity in the research process
Where can protocols be accessed? Few options to register protocols; repeated calls for protocol registration... http://www.crd.york.ac.uk/prospero/ Launched Feb 2011 April 16, 2012: 448 registered protocols from 27 countries 61 from Canada
Where can protocols be accessed? Paucity of protocol production & publication <50% (of non-cochrane SRs) on MEDLINE report working from protocols (Moher 2007) Systematic Reviews, open-access journal launched in Feb 2012, publishes protocols (and other SR products) Theme series on importance of protocol registration
Selective reporting in primary studies Gaps in evidence of selective reporting in observational studies What we know from the randomized trial literature Major discrepancies in primary outcome reporting between protocols and trial reports: changed in 33-67% of trial reports omitted in 13-42% of trial reports introduced in 11-50% of trial reports (Dwan 2011) When trials are included in systematic reviews, the problem is magnified Do systematic reviews suffer from similar selective reporting?
Selective Reporting in SRs Of 60 Cochrane reviews (Issue 3, 2000) 47 had published protocols (Silagy 2002) 91.5% contained a major change from protocol relating to methods and selected outcomes 22% (n=64) of Cochrane reviews published over a 9- month period had discrepant outcomes from their published protocols (Kirkham 2010). 75% changed primary outcome(s) Discrepant outcomes were more likely to be significant than those not changed. Represents a fraction of the problem since majority of SRs are published in peer-reviewed journals
Barrier to publishing protocols Authors lack knowledge re: what to include in protocol Existing guidance addresses conduct Existing guidance overwhelming Solution: reporting guidelines!
Reporting Guidelines Over the last 25 years, guidance for authors preparing reports for publication have emerged www.equator-network.org CONSORT Guidance for reporting parallel-group randomized trials SPIRIT Upcoming guidance for reporting protocols of randomized trials; mirrors many CONSORT items PRISMA Reporting guidance for systematic reviews and meta-analyses Newly implemented across all EPC programs PRISMA for Protocols (PRISMA-P) Aims to mirror PRISMA items; reduce workload
Scope of PRISMA-P Primarily aimed at reporting protocols of SRs for evaluating clinical intervention efficacy [RCTs] All-encompassing/extrapolation to other SR types
Development of PRISMA-P Follows recommended EQUATOR process (Moher J Clin Epi 2010)
Initial Steps 1 Paucity of protocol registration and publication outside of select groups Extend PRISMA guidance 2 Review the literature PROSPERO group 3 Funding: AHRQ Canadian Institutes of Health Research
Pre-meeting activities 4 27 international experts (journal editors, systematic review methodologists, reporting guideline developers, systematic review funders) 5 PROSPERO Delphi (Booth 2011) 6 Source of items: PROSPERO Delphi, PROSPERO register, PRISMA, IOM CER standards, SPIRIT 7 PRISMA-P meeting: June 2011, Washington, D.C.
Face-to-face meeting 8 38 items 22 items Agreement on reporting key methodological items Disagreement/discussion/consensus on many nonscientific issues (e.g. conflict of interest) PRISMA-P Statement & Explanation and Elaboration document planned for 2012 Statement currently being reviewed by steering committee Early support (i.e. planned implementation) by a number of groups
Post-meeting activities 9 Checklist refinement currently underway 1 st major revision Oct 2011 22 items 18 items (22 including sub-items) 3 sections: Administrative information Introduction Methods 2 nd major revision March 2012 18 items in progress Phrasing and wording nuances
PRISMA-P CHECKLIST
Administrative Information Section/topic # Checklist item ADMINISTRATIVE INFORMATION Title 1a Identify the report as a protocol of a systematic review. 1b If an update of a previous review, identify as such Protocol registration 2 Review authors 3 Protocol version 4 Support 5a 5b If registered, name of the registry (e.g. PROSPERO, Joanna Briggs Institute) and registration number Name, institutional affiliation, e-mail and physical mailing address of all protocol authors; Whether the report represents the original protocol or an amendment to a previously published protocol Names and contact information of organizations providing monetary or material support for the review (e.g. funding agency, foundation, company, institution), and for review sponsor, if different from funder Role(s) and responsibilities of systematic review funders, sponsors and/or institution(s), if any, in development of review methods and the decision to submit the protocol for publication, including who will have ultimate authority over each of these activities. Contributorship 6 Contributions of protocol authors; state guarantor of protocol Reported on page #
Introduction Section/topic # Checklist item INTRODUCTION Rationale 7 Objectives 8 Describe the rationale for the review in the context of what is already known Provide an explicit statement of the question(s) the review will address with reference to participants, interventions, comparisons, outcomes (PICO), as well as study design, setting, and time frame, if applicable Reported on page #
Planned Methods Section/topic # Checklist item PLANNED METHODS Eligibility criteria 9 Specify study characteristics (e.g., PICOS, setting, time frame) to be used as criteria for eligibility for review, giving rationale Information sources Search strategy Study selection process Data management & collection 10 Describe all intended information sources (e.g., names of electronic databases, unpublished literature sources, included study authors) in the search and planned date of search 11 Present full search strategy to be used for at least one electronic database, including any planned limits (e.g., years considered, language, publication type), such that it could be repeated 12 State the process that will be used for selecting studies (e.g. abstract and full text consideration, independently, in duplicate). 13a Describe planned method of extracting data from reports (e.g., piloting forms, independently, in duplicate), any processes for obtaining and confirming data from investigators and how data will be managed. Data items 14 List and define all variables for which it is intended data will be sought (e.g., PICO, funding sources), any anticipated data assumptions and simplifications, and any anticipated prioritization of review outcomes (e.g., primary, secondary). Reported on page #
Planned methods (cont d) Section/topic # Checklist item PLANNED METHODS Risk of bias in individual studies 15 Describe anticipated methods for assessing risk of bias of individual studies (including whether this will be done at the study or outcome level), and how this information is to be used in any data synthesis. Reported on page # Data Synthesis 16a 16b Reporting biases 17 Strength of evidence Conditions under which data will be quantitatively synthesized, including any planned exploration of completeness and/or consistency (e.g. I 2 ) If data are appropriate for synthesis, describe planned summary measures, methods of synthesis, and additional analyses (e.g. sensitivity or subgroup analyses, meta-regression) 16c If quantitative synthesis is not appropriate, type of summary planned 18 Specify any planned assessment of risk of bias (e.g., publication bias across studies, selective reporting within studies) and their potential impact on overall findings if present Describe how confidence in cumulative evidence across studies will be assessed (e.g., using GRADE tool), if planned.
Next steps Will approach current PRISMA endorsers Standard endorsement text Develop & pilot etool to facilitate use by authors, peer reviewers, editors Evaluate effect of endorsement
etools Why? DM is into colour! Poor implementation to date
Steering committee David Moher, Director, Ottawa EPC Mike Clarke, Professor, Queen s University Belfast, Ireland Davina Ghersi, Expert Knowledge Development Officer, National Health and Medical Research Council, Australia Lesley Stewart, Director, Centre for Reviews and Dissemination, UK Alessandro Liberati, Professor, University of Modena, Italy Mark Petticrew, Professor, London School of Hygiene and Tropical Medicine, UK Paul Shekelle, Director, Southern California Evidence-based Practice Centre Project Coordinator: Larissa Shamseer, Ottawa EPC
EPC PRISMA-P participants Stephanie Chang, Agency for Healthcare Research and Quality (Rockville, USA) Mark Helfand, Oregon Evidence-based Practice Centre (Portland, USA) Joseph Lau, Tufts Medical Centre Evidence-based Practice Centre (Boston, USA) Kathleen Lohr, Research Triangle Institute-University of North Carolina Evidence-based Practice Centre (Research Triangle Park, USA) Jennifer Tetzlaff, Ottawa Evidence-based Practice Centre (Ottawa, Canada) Thomas Trikalinos, Tufts Medical Centre Evidence-based Practice Centre (Boston, USA) Evelyn Whitlock, Oregon Evidence-based Practice Centre and United States Preventive Services Task Force (Portland, USA)
Other PRISMA-P Participants Douglas G Altman, DSc, Centre for Statistics in Medicine (Oxford, UK) Alison Booth, Centre for Reviews and Dissemination, University of York (York, UK) An-Wen Chan, University of Toronto (Toronto, Canada) Tammy Clifford, Canadian Agency for Drugs and Technologies in Health (CADTH) Peter Gøtzsche, Nordic Cochrane Centre (Copenhagen, Denmark) Jeremy Grimshaw, Ottawa Hospital Research Institute (Ottawa, Canada); KT Canada Trish Groves, British Medical Journal (London, UK) Toby Lasserson, Cochrane Editorial Unit (London, UK) Jessie McGowan, University of Ottawa (Ottawa, Canada) Melissa Norton, PLoS Medicine (London, UK) Iveta Simera, EQUATOR Network (Oxford, UK) Bill Summerskill, the Lancet (London, UK)
Questions?