Description
Correct standard errors for correlated errors within groups by clustering at the level where treatment or shared shocks vary, so false positives stay near the nominal rate.
Ordinary standard errors assume independent observations, but users within a team, sessions within a user or accounts within a region share unobserved influences. Cunningham's Monte Carlo with 1,000 draws shows ordinary least squares on clustered data giving intervals that are too narrow and far more than 5 per cent false rejections, while clustered errors restore the correct rate. Heteroskedasticity-robust errors should be the default, with clustering at the level at which treatment or aggregate variables vary. Clustering is essential in panels, repeated cross-sections such as difference-in-differences, and experiments randomised above the individual level. With about thirty clusters or fewer, even clustered errors over-reject, so use a wild cluster bootstrap or randomisation inference.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Use heteroskedasticity-robust standard errors as a default.
- Work out how variables were constructed and at what level treatment and aggregate variables vary (session, user, account, team, region).
- Cluster standard errors at that level; in panel and difference-in-differences data always cluster by unit or group, and consider multi-way clustering.
- Check the number of clusters; with about thirty or fewer, use a wild cluster bootstrap or randomisation inference.
- Report the clustering level and the number of clusters alongside the estimates.
Sources
- Causal Inference: The Mixtape (Cunningham, 2021)
Also known as
- Clustered standard errors
- Cluster-robust variance estimation
Categories
- Analytics & Behaviour
- Statistics & Modelling