Description
Balances treated and untreated users on observed covariates via their estimated probability of treatment, approximating a randomised comparison from observational data.
Listed in the treatise's causal inference curriculum and its borrowing matrix, propensity score methods estimate each unit's probability of receiving treatment given observed covariates, then match, stratify or weight so that treated and comparison groups are balanced. Rosenbaum and Rubin showed that conditioning on the propensity score suffices to remove bias from observed confounders. Its limit is fundamental: it cannot correct for unobserved confounders, so sensitivity analysis (for example Rosenbaum bounds or E-values) is essential, and covariates must be chosen with a causal model, avoiding post-treatment variables. In UX it suits comparisons between users who did and did not adopt a feature when an experiment was not run.
Attributes
- Discipline
- Experimentation & Causal Inference
- Loss type
- Unmigrated
- Goal
- Prove causation
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Define treatment, outcome and pre-treatment covariates using a DAG.
- Estimate propensity scores with logistic regression or a flexible model.
- Match, stratify or weight, and check covariate balance (standardised differences below 0.1).
- Trim units outside common support.
- Estimate the effect on the balanced sample.
- Run sensitivity analysis for unmeasured confounding and report it.
Sources
- Lost Methods in UX — A Complete Treatise (Gopinath)
- Rosenbaum PR & Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41-55
- Austin PC (2011) An introduction to propensity score methods. Multivariate Behav Res
- causalgraphicalmodels
- dowhy
- causalml
- econml
- statsmodels
- Rosenbaum, P. R. & Rubin, D. B. (1983) The central role of the propensity score in observational studies for causal effects. Biometrika
- Hernán, M. A. & Robins, J. M. (2020) Causal Inference: What If
- King, G. & Nielsen, R. (2019) Why propensity scores should not be used for matching. Political Analysis
- The Inquiry Series, C5 — Causal Inference in Python — DAGs, Target Trial Emulation & Sensitivity Analysis
Also known as
- PSM
- Propensity score weighting
- Inverse probability weighting
- Propensity matching
- Propensity score matching
Categories
- Analytics & Behaviour
- Experimentation & Causal Inference