Bayesian Structural Time Series (CausalImpact)

Description

Forecasts what a metric would have done without an intervention using a state-space model with control series, giving posterior intervals for the effect.

BSTS models a time series with components for local trend, seasonality and a regression on control series not affected by the intervention, with spike-and-slab priors selecting relevant controls. Trained on the pre-period, it produces a counterfactual forecast for the post-period; the difference from observed data, with full posterior intervals, estimates the effect cumulatively and pointwise. Key assumptions are that control series are unaffected by the intervention and that the relationship between treated and control series is stable. It is accessible through the CausalImpact package, but rigorous use requires checking pre-period fit and assumption validity rather than running defaults.

Attributes

Loss type
Latent
Stage
Post-launch
Effort
Days

Steps

  1. Choose the target series and candidate control series that are predictive but unaffected by the change.
  2. Fit the model on a long pre-period and inspect one-step-ahead prediction accuracy.
  3. Run the post-period counterfactual and inspect pointwise and cumulative effects with credible intervals.
  4. Test robustness with placebo interventions in the pre-period and alternative control sets.
  5. Check that control series did not shift at the intervention date.
  6. Report effect estimates with intervals and model diagnostics.

Sources

  • Brodersen et al., Inferring causal impact using Bayesian structural time-series models (2015), Annals of Applied Statistics
  • Scott & Varian, Predicting the present with Bayesian structural time series (2014), International Journal of Mathematical Modelling and Numerical Optimisation

Also known as

  • BSTS
  • CausalImpact

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference