Uplift Modelling

Description

Predicts the individual-level causal effect of an intervention from randomised data, to target users who will respond because of it, not regardless of it.

Response models predict who will convert; uplift models predict who will convert because they were treated. Users fall into four groups: persuadables, sure things, lost causes and 'sleeping dogs' who react negatively to treatment. Using randomised treatment and control data, uplift models (two-model approach, class transformation, uplift trees, meta-learners such as T-, S- and X-learners) estimate conditional treatment effects. Evaluation uses Qini or uplift curves on held-out data, not accuracy. In UX, uplift modelling identifies who should receive nudges, onboarding help or retention offers, and who would be harmed or annoyed by them.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Weeks

Steps

  1. Run a randomised experiment of the intervention with a sufficiently large control group.
  2. Prepare pre-treatment features for each user, avoiding any post-treatment variables.
  3. Fit uplift models (e.g. two-model baseline, X-learner, uplift random forest) with cross-validation.
  4. Evaluate with Qini coefficient and uplift curves on holdout data, comparing to random targeting.
  5. Identify segments with negative uplift (sleeping dogs) and examine them qualitatively.
  6. Deploy targeting with a persistent randomised holdout to confirm incremental impact.

Sources

  • Radcliffe & Surry, Real-world uplift modelling with significance-based uplift trees (2011), Stochastic Solutions white paper
  • Lo, The true lift model (2002), ACM SIGKDD Explorations
  • Künzel et al., Metalearners for estimating heterogeneous treatment effects (2019), PNAS

Also known as

  • Uplift modeling
  • True lift modelling
  • Incremental response modelling
  • Net lift

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning