Backdoor adjustment

Description

Using a causal diagram to identify the minimal set of confounders that must be adjusted for to estimate a causal effect from observational UX data.

When randomisation is impossible, a causal effect can still be identified if all backdoor paths (non-causal paths from treatment to outcome through common causes) are blocked. Pearl's backdoor criterion states that a set of variables is sufficient if it blocks every backdoor path and contains no descendant of the treatment. Drawing a DAG makes the assumptions explicit and shows which variables to adjust for and, equally important, which not to: adjusting for mediators removes part of the effect, and adjusting for colliders creates bias. Rigour comes from drafting the DAG from domain and qualitative knowledge before seeing outcome data, deriving the adjustment set algorithmically (dowhy or DAGitty), and stress-testing with sensitivity analysis.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Weeks

Steps

  1. State the causal question precisely: treatment, outcome, population and time window.
  2. Draw a DAG with domain experts and qualitative evidence, including unmeasured variables, before analysing outcomes.
  3. Use dowhy or DAGitty to derive minimal sufficient adjustment sets and test implied conditional independences in the data.
  4. Check that no adjustment variable is a mediator or collider, or a descendant of the treatment.
  5. Estimate the effect conditional on the adjustment set (regression, matching, weighting or doubly robust estimation).
  6. Run refutation tests (placebo treatment, random common cause) and an E-value sensitivity analysis for unmeasured confounding.

Sources

  • causalgraphicalmodels
  • dowhy
  • causalml
  • econml
  • statsmodels
  • Pearl, J., Glymour, M. & Jewell, N. P. (2016) Causal Inference in Statistics: A Primer
  • Pearl, J. (2009) Causality, 2nd ed.
  • Hernán, M. A. & Robins, J. M. (2020) Causal Inference: What If
  • The Inquiry Series, C5 — Causal Inference in Python — DAGs, Target Trial Emulation & Sensitivity Analysis
  • DAGs+SEM Playbook (Vivek Gopinath)
  • Causal Inference: What If (Miguel Hernán & James Robins)
  • Statistical Rethinking (Richard McElreath)
  • Causality (Judea Pearl)
  • DAGs+SEM Playbook (Gopinath)
  • Causal Inference (Hernán & Robins)
  • Statistical Rethinking (McElreath)
  • The Inquiry Series, Ch 46 — Directed Acyclic Graphs — Making Assumptions Explicit
  • Cinelli, C., Forney, A. & Pearl, J., A Crash Course in Good and Bad Controls (2022)

Also known as

  • Backdoor criterion
  • Confounder adjustment via DAG
  • Backdoor adjustment
  • Backdoor and front-door criteria

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference