Description
Uses honest random forests to estimate how treatment effects vary across users, with valid confidence intervals and tests for heterogeneity.
Causal forests adapt random forests to estimate conditional average treatment effects. Splits are chosen to maximise treatment effect heterogeneity, and 'honesty' uses separate subsamples for choosing splits and estimating effects, yielding asymptotically valid confidence intervals. Built-in diagnostics include calibration tests for heterogeneity and best linear projections of effects onto covariates. Compared with ad hoc segment slicing of A/B results, which inflates false discoveries, causal forests provide disciplined discovery of who benefits and who is harmed, and can work with observational data under unconfoundedness with orthogonalisation.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Unmigrated
- Goal
- Prove causation, Segment users
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Weeks
Steps
- Start with randomised experiment data (preferred) or observational data with plausible unconfoundedness.
- Select pre-treatment covariates that might moderate effects, e.g. tenure, device, prior usage.
- Fit a causal forest (grf package) with honest splitting and sufficient trees (e.g. 2,000+).
- Run the calibration test for heterogeneity and compute the average treatment effect with confidence interval.
- Summarise heterogeneity with best linear projection and by ranking users into quantiles of predicted effect (RATE or TOC curves).
- Validate key subgroups in a follow-up experiment before acting.
Sources
- Wager & Athey, Estimation and inference of heterogeneous treatment effects using random forests (2018), JASA
- Athey, Tibshirani & Wager, Generalized random forests (2019), Annals of Statistics
Also known as
- Causal forests
- Generalised random forests
- GRF
- Heterogeneous treatment effect estimation
Categories
- Analytics & Behaviour
- Computational & Machine Learning