White Paper January 2017 META-ANALYSIS FOR HEALTH TECHNOLOGY SUBMISSIONS WORLDWIDE: A REPORT CHECKLIST FOR BEST PRACTICE. Sarah Batson, Neil Webb

Transcription

1 05 NETWORK White Paper January 2017 META-ANALYSIS FOR HEALTH TECHNOLOGY SUBMISSIONS WORLDWIDE: A REPORT CHECKLIST FOR BEST PRACTICE Sarah Batson, Neil Webb

2 Network meta-analysis (NMA) is an accepted statistical method used to model the clinical- and cost-effectiveness of treatments in the absence of data from head-to-head randomised controlled trials (RCTs). 1, 2 In the fifth of our DRG investigative series, we assess submission guidelines by health technology assessment (HTA) bodies and suggest a reporting checklist for NMAs to support manufacturers in meeting the requirements for global technology appraisal. THE QUESTION What advantages do NMAs offer in the modelling of clinical evidence? Although results from head-to-head RCTs represent the accepted gold standard for clinical evidence synthesis, NMA is a recommended technique where these data are unavailable. 2 Indirect comparisons (ICs) and NMAs are acknowledged methodologies by HTA agencies worldwide including the National Institute for Health and Care Excellence (NICE), the Canadian Agency for Drugs and Technologies in Health (CADTH), the French Haute Autorite de la Sante (HAS) and the Pharmaceutical Benefits Advisory Committee in Australia (PBAC), as well as emerging national agencies in Austria, Brazil, Colombia, Cuba, and Ireland. 3 The main advantage of NMA is the ability to synthesise all available evidence from an extensive treatment network and estimate the efficacy of each treatment versus all comparators in a single analysis. 4 Based on our review of submission guidelines by HTA agencies worldwide, we have created a reporting checklist for NMAs to support manufacturers in meeting the requirements for technology appraisal. 2

3 THE RESEARCH Identification of guidance for the methods and reporting of ICs and NMAs. An invaluable source of country-specific pharmacoeconomic (PE) guidelines is the webbased repository maintained by the International Society of Pharmacoeconomics and Outcomes Research (ISPOR). This resource includes PE recommendations, PE guidelines, and submission guidelines from 38 countries. In July 2015, we searched the ISPOR website and the recently published PRISMA-NMA extension for guidance relating to the methods and reporting of IC and NMA. We checked reference lists of the identified guidelines and hand-searched for any additional guidelines or method reviews via the websites of each HTA body identified on the ISPOR web-based repository. Non-English language guidelines were excluded. We used the data identified through our searches to create a current checklist to aid the reporting of NMA analyses specifically for HTA purposes. Country-specific pharmacoeconomic guidelines maintained by ISPOR N=38 Australia, Austria, Baltic states, Belgium, Brazil, Canada, China, Colombia, Croatia, Cuba, Denmark, Egypt, England and Wales, Finland, France, Germany, Hungary, Ireland, Israel, Italy, Malaysia, Mexico, New Zealand, The Netherlands, Norway, Poland, Portugal, Russian Federation, Scotland, Slovak Republic, Slovenia, South Africa, South Korea, Spain, Sweden, Thailand, Taiwan, and the United States Not available in English N=10 Brazil, China, Colombia, Cuba, Hungary, Mexico, Slovenia, South Korea, Spain Hand-searched guidelines/method reviews N=8 Brazil, Canada, England and Wales (n=2) ), Ireland, EUnetHTA, PRISMA, Scotland No mention of IC/NMA N=16 Austria, Baltic states, Brazil, Denmark, Egypt, Finland, Italy, Israel, Malaysia, The Netherlands, Portugal, Russian Federation, Slovak Republic, Taiwan, and Thailand Guidelines/method reviews with reference to IC/NMA N=20 Australia, Belgium, Canada (n=2), Croatia, England and Wales (n=3), EUnetHTA, France, Germany, Ireland (n=2), New Zealand, Norway, Poland, PRISMA, Scotland, South Africa, and the United States Guidelines which make reference only to the use of IC/ NMA N=5 Croatia, France, Norway, Poland, and the United States Guidelines with detailed information regarding the use of IC/NMA N=15 Australia, Belgium, Canada (n=2), England and Wales (n=3), EUnetHTA, Germany, Ireland (n=2), New Zealand, PRISMA, Scotland and South Africa EUnetHTA, European network for health technology assessment; IC, indirect comparison; ISPOR, The International Society of Pharmacoeconomics and Outcomes Research; NMA, network meta-analysis; PRISMA, Preferred reporting items for systematic reviews and meta-analyses; Number includes the two additional documents from NICE detailing guidelines for multiple technology appraisals and single technology appraisals; An additional method review was identified on the CADTH website that supplemented the guidelines listed on the ISPOR web-based repository; The link to the NICE guidelines on the ISPOR web-based repository lead to an out of date set of guidelines, the current guidelines (2013) were hand searched on the NICE website; An additional guideline for evaluating the clinical effectiveness in health technologies was available on the Health Information and Quality Authority website; The link to the SMC guidelines on the ISPOR web-based repository lead to an out of date set of guidelines, the current guidelines (2014) were hand searched on the SMC website. 3

4 THE FINDINGS Summarising the data to create a best practice checklist Our searches identified 15 publications, including guidance documents from nine countries and guidelines from two collaborations (PRISMA and EUnetHTA). The PRISMA statement focuses primarily on RCT data, aiming to guide authors on the preferred reporting methods for SRs and NMAs. EUnetHTA promotes good practice in HTA processes by collaborating with several HTA bodies across Europe (including some countries that were excluded in this study) to create transferable guidelines. Several of the reviewed guidance documents (n=16) provided limited guidance for meta-analysis and did not comment specifically on ICs or NMA. These documents were from Austria, Baltic states, Brazil, Denmark, Egypt, Finland, Italy, Israel, Malaysia, The Netherlands, Portugal, Russian Federation, Slovak Republic, Taiwan, and Thailand and were not used to inform the checklist. Limitations of our research As with all studies, our research was subject to some limitations. For example, although the search followed a structured approach, it may not be considered systematic. However, an extensive search strategy would be needed to conduct country-specific database searches and would be technically unfeasible. Our study was also limited by the availability and reporting of guidelines. We only considered publications available in English. A further investigation would be needed to ensure the checklist is suitable for the countries where guidelines are not reported in English. Comparisons with previous research A previous review 5 explored the guidelines listed on the ISPOR web-based repository with the aim of summarising the information regarding the use of NMA for submission to HTA bodies. We consider our study to be more comprehensive than the former review as websites of HTA bodies in each country listed on the ISPOR web-based repository were hand searched for further publications regarding IC and NMA. We also ensured that documents listed on the ISPOR web-based repository were the most recent publications and, therefore, the most current, publically available guidelines at the time were used. In June 2015, an extension to the PRISMA statement was published incorporating the reporting of NMA alongside systematic reviews. 6 The extension statement added five items to the PRISMA checklist and included: reporting of a network of evidence; descriptions of the geometry of the network, statistical methods used and patient and trial characteristics to allow comparison of the trials within the network; an investigation of inconsistency in networks. We reviewed the updated PRISMA checklist alongside the country-specific HTA guidelines to inform our comprehensive checklist. Although the NMA extension statement is a recent publication, our study provides readers with a practical checklist to use specifically for NMA in HTA submissions rather than general SR and NMA reports. There are many crossovers between the NMA extension statement and our checklist; however, the addition of the country-specific HTA guidelines ensures the checklist is fit for global HTA submissions. 4

5 RECOMMENDATIONS The methodological quality and reporting of NMAs in manufacturers submissions to HTA bodies needs to be of a high standard to facilitate a positive recommendation The use of an appropriate checklist helps manufacturers determine whether an NMA report meets the requirements of HTA bodies worldwide Our checklist provides practical support for assessing the suitability of NMA reports for use in HTA submissions Our checklist can be used as a quality tool to critically appraise the reporting of NMAs in HTAs Checklist for the suitability of an NMA report for HTA submissions based on national guidelines, adapted from Laws et al, Data Bucher method was not properly conducted for most analyses Have the raw data for the analyses from each study been presented including the source of the data? Description of patient and treatment characteristics provided? Has it been noted if any eligible treatments have been grouped (with justification)? Has a visual presentation of the evidence network been provided alongside a tabulated version? Has the geometry of the network been explored including potential sources of bias? Statistical Methodology Have the statistical methods been described? Has the software package been specified? If Bayesian, have the prior distributions been described? If Bayesian, has the assessment of convergence been described? Has the choice of random or fixed-effects models been described? Where random effect models are used have measures of between-study heterogeneity been reported? Has an appropriate measure of relative treatment effect been used and described? Analyses Performed Has a base case analysis been presented and clearly defined? Has the rationale for sensitivity analyses been described? Presentation of Results Are results for relative treatment effects presented in tabular and forest plot form? Have the relative treatment effects been described? Have treatment rankings been reported as probabilities and median ranks? Have rank probabilities been summarised graphically? Have the results of the sensitivity analyses been described? Have sensitivity analyses been reported separately from the base case? Technical issues Has meta-regression been performed and described? If relevant has inconsistency been explored? Have the homogeneity of direct comparisons been assessed (e.g. I 2 ) 5

6 IN SUMMARY We used evidence from 15 publications, (including guidance documents from nine countries and guidelines from PRISMA and EUnetHTA) to create a checklist for the methods and reporting of NMAs. Our checklist provides practical support to manufacturers enabling them to assess the suitability of NMA reports in meeting the requirements of global HTA bodies. Our reporting checklist can also serve as a valid quality tool to critically appraise the reporting of NMAs in HTAs. 6

7 IF YOU NEED TO KNOW MORE Our multi-disciplinary team of systematic review analysts, statisticians and medical writers has extensive experience in the design, conduct and communication of systematic literature reviews and metaanalyses. We are dedicated to delivering high-quality systematic reviews and statistical analyses, and attention-to-detail and accuracy underpin all that we do. Gathering all the relevant data takes time and energy. We use industry-standard techniques and approaches to ensure we minimise errors and get it right first time ; to this end, we have developed a customised systematic review software to manage our data review. Contact Stephen Mitchell at: smitchell@teamdrg.com or Sarah Batson at: sbatson@teamdrg.com REFERENCES 1. Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol Jun;50(6): Dias S, Welton, N.J., Sutton, A.J. & Ades, A.E. NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials Tan SH, Bujkiewicz S, Sutton A, Dequen P, Cooper N. Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons. J Health Serv Res Policy Oct;18(4): Jansen JP. Network meta-analysis of survival data with fractional polynomials. BMC Med Res Methodol. 2011;11: Laws A, Kendall R, Hawkins N. A comparison of national guidelines for network meta-analysis. Value Health Jul;17(5): Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta- analyses of Health Care Interventions: Checklist and Explanations. Ann Intern Med Jun 2;162(11):

8 Sarah Batson, Consultant Statistician Sarah has extensive experience in meta-analysis, pair-wise metaanalysis, indirect comparisons, and network-meta-analysis. She can perform statistical analyses in R, Stata and WinBUGs and has worked on several successful submissions to HTA bodies including NICE and the SMC. Sarah holds a BSc in Biochemistry, an MSc in Medical Statistics, and a PhD in Biological Sciences. Neil Webb, Consultant Neil has worked on numerous systematic reviews across many diseases and medical devices. He is proficient at developing search strategies, technical data extraction, report writing as well as presenting the key findings of systematic reviews to clients. Neil holds a BSc in Medical Genetics. CONTACT 6 Talisman Business Centre, Bicester, Oxfordshire OX26 6HR +44 (0) Access@TeamDRG.com