Disproportionality statistics for signal detection

Transcription

1 Disproportionality statistics for signal detection Andrew Bate Senior Director, Analytics Team Lead, Epidemiology ISOP-UMC Training Course Mysore, 13 January 2015

2 Disclosures I am a full time employee of Pfizer and hold stocks and stock options

3 Spontaneous reporting as a tool for post marketing surveillance Analysis of spontaneous reports of suspected adverse drug reactions (ADRs) is a valuable tool in the detection of previously unknown drug adverse reactions Reports of adverse events (AEs) associated with a drug are not necessarily true ADRs, that is, they may be temporally associated with a drug but not caused by the drug. Hypothesis generation of new possible side effects from such data is referred to as signal detection Ref Bate and Evans 2009 PDS

4 Signal of suspected causality - definition Information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verificatory action. Ref CIOMS VIII Practical Aspects of Signal Detection in Pharmacovigilance 2009

5 Spontaneous reporting limitations Often limited clinical information on reports and satisfactory secondary case evaluation is not always possible Not all ADRs that occur will be recognized as drug induced by a healthcare professional Even those that are suspected will not necessarily be reported Suspicion can mistakenly rest on the drug, coincidental spontaneous ADR case reports resulting Control information is not collected as part of spontaneously reported systems, the drug use is not known, and there is no direct information on disease incidence Ref Bate et al 2008 FCP

6 Quantitative signal detection To detect potential signals for further investigation that are not readily recognisable on a single case report nor otherwise readily apparent at case entry Enhance rather than replace other methods of signal detection E.g. Initial case review of spontaneous reports Overall help to detect signals earlier, more effectively and more efficiently Primarily useful in large data sets

7 Quantitative signal detection different approaches Several methods for screening spontaneously reported data Most common approach: measures of disproportionality Different measures of disproportionality But all aim to detect the unexpectedly frequently reported relative to a background of other reports Quantitative outputs clinically reviewed and some subsequently considered as signals

8 Metrics used in quantitative screening of spontaneous reports 2 by 2 contingency table AE of interest (y) Other AEs Drug of interest (x) a b Other drugs c d Proportional Reporting Ratio (PRR) = [a (a + b)] / [c (c + d)] Reporting Odds Ratio (ROR) = [a b] / [c (d)] Observed / Expected 1 = 1 Frequentist basis of EBGM and IC 8 [a (a + b)] / [(a + c) (a + b + c + d)] See Bate and Evans 2009 PDS, for an overview

9 Bayesian methods in quantitative signal detection - Bayesian methods give framework for how to combine prior information with data - Prior information/knowledge + data - Account in a conceptually transparent way for external data and/or subjective opinion - Bayesian methods - not always more complex IC log 2 Observed Expected + + 1/ 2 1/ 2 Refs Noren et al 2013, for IC formula Bate and Evans 2009 PDS, for an overview - High Observed and Expected, shrinkage no practical impact - When Expected and Observed are low, IC tends to zero - Note: Idea is that overall the estimates will be better not that each estimate will necessarily be better Informative priors also possible

10 Change in Information Component over time Captopril - Coughing IC -2 79:1 81:1 83:1 85:1 87:1 89:1 91:1 93:1 95:1 Time(year)

11 Change in Information Component over time 6 Digoxin - Rash IC -6 67:1 70:1 73:1 76:1 79:1 82:1 85:1 88:1 91:1 94:1 Time(year)

12 Change in Information Component over time Digoxin - Acne IC -6 67:1 70:1 73:1 76:1 79:1 82:1 85:1 88:1 91:1 94:1 Time(year)

13 Change in Information Component over time Practolol, ATC C07AB - Peritonitis sclerosing Practolol ATC C07AB year

14 Differences in metrics The best metric No single best method PRR arguably most easy to interpret Shrinkage in Bayesian approaches advantageous at low counts, but more complex Van Puijenbroek et al 2001 Underreporting less effect for ROR than PRR Van Der Heijden 2002, Rothman et al 2004 ROR gives more volatile scores 14

15 Choice of metrics ROR =4.24, PRR= 3.59, O/E= 3.33

16 Choice of metrics ROR =3.88, PRR= 3.59, O/E= 3.33

17 Ongoing Misunderstanding The PRR of 6 indicates that for this drug the risk of reporting this event is six times higher compared with reference drugs. Should be: The PRR of 6 indicates that for this drug the probability of reporting this particular event rather than any other event is six times higher compared to the probability for reference drugs 17

18 Background reading Bate A et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998; 54: Bate A, Lindquist M, and Edwards IR. The application of knowledge discovery in databases to postmarketing drug safety: example of the WHO database. Fundam Clin Pharmacol, (2): p Bate A, Evans SJW Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiology and Drug Safety. 18(6): DuMouchel, W. (1999). Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 53(3), Evans, S. J. W., Waller, P. C., & Davis, S. (2001). Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiology and drug safety, 10(6), Gipson, G. (2012). A shrinkage based comparative assessment of observed to expected disproportionality measures. Pharmacoepidemiology and drug safety, 21(6), Lindquist M et al. From association to alert - A revised approach to international signal analysis. Pharmacoepidemiology and Drug Safety 1999; 8: S15-S25 Norén GN, Hopstadius J, Bate A Shrinkage observed to expected ratios for robust and transparent large scale pattern discovery. Statistical Methods in Medical Research. 22 (1), van Puijenbroek EM et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for Adverse Drug Reactions. Pharmacoepidemiology and Drug Safety 2002; Volume 11( 1) pp

19 Conclusions Several methods for screening spontaneously reported data Most common approach: measures of disproportionality Different measures of disproportionality But all aim to detect the unexpectedly frequently reported relative to a background of other reports No one best method Quantitative outputs clinically reviewed and some subsequently considered as signals Quantitative signal detection needs to be part of robust overall signal management programme