Propensity Score Matching

Description

Balances treated and untreated users on observed covariates via their estimated probability of treatment, approximating a randomised comparison from observational data.

Listed in the treatise's causal inference curriculum and its borrowing matrix, propensity score methods estimate each unit's probability of receiving treatment given observed covariates, then match, stratify or weight so that treated and comparison groups are balanced. Rosenbaum and Rubin showed that conditioning on the propensity score suffices to remove bias from observed confounders. Its limit is fundamental: it cannot correct for unobserved confounders, so sensitivity analysis (for example Rosenbaum bounds or E-values) is essential, and covariates must be chosen with a causal model, avoiding post-treatment variables. In UX it suits comparisons between users who did and did not adopt a feature when an experiment was not run.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Days

Steps

  1. Define treatment, outcome and pre-treatment covariates using a DAG.
  2. Estimate propensity scores with logistic regression or a flexible model.
  3. Match, stratify or weight, and check covariate balance (standardised differences below 0.1).
  4. Trim units outside common support.
  5. Estimate the effect on the balanced sample.
  6. Run sensitivity analysis for unmeasured confounding and report it.

Sources

  • Lost Methods in UX — A Complete Treatise (Gopinath)
  • Rosenbaum PR & Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41-55
  • Austin PC (2011) An introduction to propensity score methods. Multivariate Behav Res
  • causalgraphicalmodels
  • dowhy
  • causalml
  • econml
  • statsmodels
  • Rosenbaum, P. R. & Rubin, D. B. (1983) The central role of the propensity score in observational studies for causal effects. Biometrika
  • Hernán, M. A. & Robins, J. M. (2020) Causal Inference: What If
  • King, G. & Nielsen, R. (2019) Why propensity scores should not be used for matching. Political Analysis
  • The Inquiry Series, C5 — Causal Inference in Python — DAGs, Target Trial Emulation & Sensitivity Analysis

Also known as

  • PSM
  • Propensity score weighting
  • Inverse probability weighting
  • Propensity matching
  • Propensity score matching

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference