Interrupted Time Series (ITS)

Description

Models a long series of measurements before and after a change to estimate immediate level shifts and slope changes beyond the pre-existing trend.

Named in the treatise's borrowing matrix among epidemiology's key methods, ITS uses many time points before and after an intervention to estimate whether it changed the level and trend of an outcome, controlling for the pre-existing trend, seasonality and autocorrelation. It is one of the strongest quasi-experimental designs when no control group exists, and adding a comparison series makes it stronger. For UX, it evaluates releases, policy changes and outages with daily or weekly metrics such as support contacts, error rates or task completion. Typical requirements are at least 8-12 points per segment and attention to concurrent events that could explain changes.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Days

Steps

  1. Define the intervention date and outcome series at consistent intervals.
  2. Collect sufficient pre- and post-intervention points (ideally 12 or more each).
  3. Fit segmented regression with terms for baseline trend, level change and slope change.
  4. Model seasonality and autocorrelation (for example ARIMA errors).
  5. Add a control series or outcome if available and check for co-interventions.
  6. Report level and slope changes with CIs and visualise the counterfactual trend.

Sources

  • Lost Methods in UX — A Complete Treatise (Gopinath)
  • Wagner AK et al. (2002) Segmented regression analysis of interrupted time series studies. J Clin Pharm Ther 27
  • Bernal JL, Cummins S & Gasparrini A (2017) Interrupted time series regression for the evaluation of public health interventions. Int J Epidemiol
  • Wagner et al., Segmented regression analysis of interrupted time series studies in medication use research (2002), Journal of Clinical Pharmacy and Therapeutics
  • Bernal, Cummins & Gasparrini, Interrupted time series regression for the evaluation of public health interventions: a tutorial (2017), International Journal of Epidemiology
  • Shadish, Cook & Campbell, Experimental and Quasi-Experimental Designs (2002)
  • Gratton, C. & Jones, I., Research Methods for Sports Studies (2010)

Also known as

  • ITS
  • Segmented regression
  • Interrupted time series analysis
  • Event study (single series)

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference