Use of decision modelling in economic evaluations of diagnostic tests: an appraisal of Health Technology Assessments in the UK since 2009

Transcription

1 Use of decision modelling in economic evaluations of diagnostic tests: an appraisal of Health Technology Assessments in the UK since 2009 Yaling Yang, Lucy Abel, James Buchanan, Thomas Fanshawe, Bethany Shinkins Methods for Evaluating Medical Tests and Biomarkers (MEMTAB) Symposium 2016, Birmingham

2 Background Economic evaluation of health interventions Comparative analysis of both COST and OUTCOMES Cost per Quality Adjusted Life Years (QALYs) Incremental Cost per QALY gained (ICER) Decision analytic modelling Simulate downstream consequences using mathematical relationships Facilitate comparisons of alternative scenarios Inputs Consequences (Cost & QALYs) & probabilities (likelihood) Expected cost & QALYs Useful under circumstances: RCTs are not feasible or practical; Trials only measures intermediate endpoints or short-term follow up; Need to compare a big number of scenarios or strategies; Need to synthesize best available data from different sources

3 Background Decision modelling: promising way to evaluate diagnostic tests Evaluation under broader framework: test treatment process Link test accuracy to patient health outcome Compare various combinations of test results and treatment decisions Synthesize data from diagnostic test, treatment effects & epidemiological studies Aims: To provide a methodological review & critical appraisal of model-based cost effectiveness of diagnostic tests

4 Methods Focus on UK Health Technology Assessment (HTA) ( ) Identify model-based economic evaluation of diagnostic tests Extract data using a quality checklist and associated response system ( met, partially met and not met ) Summarize data for all tests, and by test types, disease areas and over time.

5 General findings Forty HTA reports were included in the appraisal (out of a total of 515). Six main categories of tests: imaging (17), lab-based (7), genetic (6 reports), point of care (6), clinical decision rules (2), and others (2, surgical and questionnaire). 21 studies defined another test as a reference standard, whereas 17 used clinical criteria or clinical follow up as sole or part of reference, and 2 had none. All analyses adopted the perspective of the NHS or NHS and social care. A decision tree was the most commonly used model, followed by Markov and discrete event simulation models. Most models used cost per QALY analysis, with cost per case detected used by 2 models. Time horizon tended to be lifetime with some using durations below 5 years, or only covering the testing period.

6 Modelling quality checklist (modified from Philips 2006) No. Checklist categories Number of questions in each category (N=39) 1 Decision problem and scope specified 5 2 Identification and description of comparators 3 3 Appropriate data identification 1 4 Sufficient detail for data incorporation 4 5 Quality and incorporation of test accuracy data 3 6 Quality and incorporation of treatment data 2 7 Source and incorporation of cost data 2 8 Source and incorporation of utility data 2 9 Appropriate model structure and assumptions 7 10 Sufficient examination of uncertainty 7 11 Sufficient examination of model validity 3

7 Modelling quality checklist (1= met, 0.5= partially met, 0= not met ) (Modified from Philips 2006) ision problem and scope specified 1 Is there a clear statement of the decision problem? 2 Is the perspective of the model stated clearly? 3 Has the target population been identified? 4 Are the model inputs consistent with the stated perspective? 5 Are the primary outcomes of the model consistent with the perspective, scope and overall objective of the model? tification and description of comparators 6 Have all feasible and practical options been identified? 7 Have the comparators being evaluated been clearly described? 8 If comparators have been excluded from the evaluation, have these exclusions been justified? ropriate data identification 9 Are the data identification methods transparent, systematic and appropriate given the objectives of the model? ficient detail for data incorporation 10 Have all data incorporated into the model been described and referenced in sufficient detail? 11 Where choices have been made between data sources, are these justified appropriately? 12 Are transition probabilities calculated appropriately? 13 Has discounting been conducted? lity and incorporation of test accuracy data 14 Has the quality of the test accuracy data been assessed? 15 Have diagnostic accuracy data been derived from high quality data sources (hierarchy of evidence)? 16 Are tests in sequence treated dependently, where appropriate? lity and incorporation of treatment data 17 Has the quality of the treatment effect data been assessed? 18 Have relative treatment effects been derived from high quality data sources (hierarchy of evidence)? rce and incorporation of cost data 19 Has the source of cost data been presented clearly? 20 Have costs been inflated to a specific year, where appropriate? rce and incorporation of utility data 21 Is the source for the utility weights referenced and justified? 22 Are the utilities incorporated into the model appropriately? 23 Have the reasons behind the type of decision analytic model chosen been fully described and justified? ropriate model structure and assumptions 24 Has a systematic review of existing economic evaluations been carried out? 25 Is the structure of the model consistent with a coherent theory of the health condition under evaluation? 26 Are the structural assumptions underpinning the model transparent and justified? 27 Have the methods used to extrapolate short-term results to final outcomes been documented and justified? 28 Has the time horizon been stated and justified? 29 Has cycle length of Markov models been justified? 30 Has parameter uncertainty been addressed via sensitivity analysis? ficient examination of uncertainty 31 Has probabilistic sensitivity analysis been carried out. If not, has this omission been justified? 32 If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified? 33 If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified? 34 Have structural uncertainties been addressed via sensitivity analysis? 35 Have alternative assumptions related to final outcomes been explored through sensitivity analysis? 36 Has value of information analysis been carried out? 37 Has the face validity been reviewed by someone external to the model developers? ficient examination of model validity 38 Has the mathematical logic of the model been assessed? (eg, using null and extreme values)

8 Proportion of responses to questions according to criteria categories (n=40) 100% 90% 80% % % % 40% 30% 20% Not met Partially met Met 10% 0%

9 Individual question/criterion of concern Number Questions Met Partially met Not met If comparators have been excluded from the evaluation, have these exclusions been 8 justified? Has the quality of the treatment effect data been assessed? Are tests in sequence treated dependently, where appropriate? Have costs been inflated to a specific year, where appropriate? Has cycle length of Markov models been justified? If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified? Have structural uncertainties been addressed via sensitivity analysis? Have alternative assumptions related to final outcomes been explored through sensitivity analysis? Has the face validity been reviewed by someone external to the model developers? Has the mathematical logic of the model been assessed? (eg, using null and extreme values) Have the model and its results been compared to the findings of other models and studies, and any disagreements or inconsistencies been explained (cross-validity)?

10 Discussion and conclusion Further validation of the checklist is needed. Models in HTA reports are of high quality Decision problem well defined Extrapolating long term outcomes logical and clinically relevant Systematic reviews of test accuracy evidence commonly conducted. Areas require improvements: Comparators excluded from analysis due to resource limitations or lack of data. Quality of test accuracy studies varied largely Uncertainty surrounding the health effects of a test ( via the treatment pathway not explored in depth or underestimated. Quality of treatment effect data not assessed in a systematic way. Highlight inherent complexity of modelling cost effectiveness of diagnostic tests, given their indirect impact on health outcome.

11 Acknowledgements Yaling Yang is funded by NIHR Oxford Biomedical research centre. Lucy Abel is funded by NIHR Oxford Diagnostic Evidence Cooperative and NIHR Research Capacity Funding Thomas Fanshawe and Bethany Shinkins were funded by NIHR National School of Primary Care during this study