Interrupted time series

Transcription

1 Antonio Gasparrini London School of Hygiene and Tropical Medicine, UK Centre for Evaluation & Centre for Statistical Methodology 22 May 2018

2 Key messages ITS is a very strong and very weak design: quasi-experimental, but requiring strong assumptions Modelling sophistication often masks key underlying flaws The ITS design does no need controls: sometimes a good idea, sometimes detrimental Erase the term causal from every ITS article (and possibly every observational study)

3 Outline 1 An example 2 The ITS design 3 ITS modelling framework 4 Design extensions 5 Controlled ITS 6 Multi-location ITS 7 Discussion

4 A motivating example Research question: Did the implementation of a state-wide smoking ban in Italy on 10 January 2005 reduce the rates of acute coronary events (ACEs)? Nice quasi-experimental setting, but: Traditional study designs (cohort or case-control) are hardly applicable, given the lack of a suitable comparison Simple comparison before vs after the ban can be affected by underlying trends in ACE rates Ideally, we would need a study design that relies on a pure within-population comparison (no external controls) and able to control for time trends

5 The interrupted TS design The interrupted time series (ITS) design is often applied in public health evaluation of interventions occurring at a specific time point ITS is often described as a quasi-experimental design: it often uses a before-after comparison accounting for underlying trends, in which a population acts as its own control Advantages: time-invariant factors are controlled by design, while modern regression methods also allow for control of time-varying confounders However, the design is based on a series of strict assumptions that need to be accounted for in the assessment of causal hypotheses

6 Interrupted series Trend of ACEs Sicily, Std rate x 100, Year

7 Definition of a counterfactual Trend of ACEs Sicily, Std rate x 100, Year

8 Change in the post-intervention period Trend of ACEs Sicily, Std rate x 100, Year

9 Regression model General ITS regression model: P g(y t) = α + f (x t, t) + βt + s(d t) + h p(z pt) + ɛ t p=1 where: f (x t) defines the impact model, dependent on parameterization of the intervention (usually binary) x t and time t t and s(d t) define the (linear) trend and (optionally) seasonality h j (z tj ) define (optional) contributions of other time-varying factors

10 Alternative impact models

11 Comparison with change-in-slope model Trend of ACEs Sicily, Std rate x 100, Year

12 Seasonality Trend of ACEs Sicily, Std rate x 100, Year

13 Non-linearity in the trend I Trend of ACEs Sicily, Std rate x 100, Year

14 Non-linearity in the trend II Tuscany, Std rate x 10, Std rate x 10,

15 Model checking: auto-correlation ACF plot without seasonal control ACF Lag ACF plot with seasonal control ACF Lag

16 More complex designs ABA Applicable in situations when intervention is stopped/reversed Less sensitive to mis-modelling of trends: more robust ABC Applicable with multi-component interventions Different steps implemented at different times Even further: controlled ITS and multi-location ITS

17 Controlled ITS

18 Types of controls Location-based controls: a different area (e.g., country, regions) Characteristic-based controls: group not targeted (e.g., sex) Behaviour-based controls: group not affected (e.g., age group) Historical-cohort controls: cohorts affected at different times (e.g., schools) Outcome-based controls: another outcome unaffected (e.g., helmet and limb injuries) Time-period controls: period unaffected (e.g., days and road lights)

19 Multi-location studies Sometimes the investigation includes multiple areas, groups, or populations, providing a setting for multi-location ITS Advantages: More representative Larger population under study, and therefore higher statistical power Study designs more robust to biases affecting standard studies Possibility to quantify and characterize heterogeneity However, the designs and statistical methods are more generally more complex

20 A two-stage meta-analytical approach Effect of ban Italian regions, Abruzzo Basilicata Calabria Campania EmiliaRomagna FriuliVeneziaGiulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana TrentinoAltoAdige Umbria ValleD'Aosta Veneto RE Model 1.03 [0.95, 1.11] 1.15 [1.02, 1.30] 0.93 [0.88, 0.98] 0.95 [0.92, 0.98] 0.97 [0.93, 1.01] 0.97 [0.90, 1.04] 0.97 [0.93, 1.01] 0.99 [0.92, 1.06] 0.99 [0.96, 1.02] 0.99 [0.92, 1.05] 0.89 [0.77, 1.03] 0.92 [0.88, 0.96] 1.00 [0.96, 1.05] 0.97 [0.90, 1.05] 0.89 [0.86, 0.93] 0.92 [0.89, 0.96] 1.02 [0.92, 1.13] 0.94 [0.85, 1.03] 0.89 [0.71, 1.12] 1.02 [0.98, 1.07] 0.97 [0.95, 0.99] RR

21 A one-stage hierarchical approach Std rate Abruzzo Trend of ACEs Italy, Basilicata Calabria Time (months) Campania riuliveneziagiuli Lazio Liguria Lombardia EmiliaRomagna Marche Molise Piemonte Puglia Sardegna Sicilia Toscana TrentinoAltoAdige Umbria ValleD'Aosta Veneto

22 With non-linearity Std rate Abruzzo Trend of ACEs Italy, Basilicata Calabria Time (months) Campania riuliveneziagiuli Lazio Liguria Lombardia EmiliaRomagna Marche Molise Piemonte Puglia Sardegna Sicilia Toscana TrentinoAltoAdige Umbria ValleD'Aosta Veneto

23 Data requirements Series of observations at equally-spaced times (but missing allowed) Enough measures both before and after the intervention Stable measurement procedures Information (measurements) of any potential time-varying confounder Optional: control series

24 Strength and limitations Strength Quasi-experimental setting: population acting as its own control Use of administratively-collected data Alternative designs for more complex interventions Use of modern regression modelling tools Limitations Sensitive to modelling choices for impact and trend Issues with measurement changes and concurrent events Ecological design mostly based on aggregated data Problems with diffuse effects (long lags)

25 Suggestions Careful definition the research question and study design Importance of substantive (epidemiological, public health) knowledge on the phenomenon under study Ispection of pre-intervention series Detailed a-priori specification of modelling approach and (limited) alternative options (Very) limited use of statistical criteria for model selection Prudent interpretation of the results

26 References Tutorial article Lopez Bernal J, Cummins S,. regression for the evaluation of public health interventions: a tutorial. International Journal of Epidemiology. 2017;46(1): [Link] Upcoming articles Lopez Bernal J, Cummins S,. The use of controls in interrupted time series studies of public health interventions. Under review. Lopez Bernal J, Soumerai S,. Model selection in interrupted time series studies. Under review. Other articles Biglan A, et al. The value of interrupted time-series experiments for community intervention research. Prevention Science. 2000;1(1): Wagner AK, et al. Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics. 2002;27: Kontopantelis E, et al. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015;350:h2750. R code and other references github.com/gasparrini