Sensitivity analysis with the E-value

Description

Quantifying how strong an unmeasured confounder would need to be to explain away an observed causal estimate from non-randomised UX data.

Every observational causal estimate rests on the assumption of no unmeasured confounding. The E-value is the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need with both the treatment and the outcome, conditional on measured covariates, to fully explain away the observed association. For a risk ratio RR above 1, E = RR + sqrt(RR x (RR minus 1)). An E-value is also computed for the confidence limit closest to the null. Large E-values (e.g. above 3) indicate robustness; values near 1 mean modest confounding could nullify the result. Rigour comes from reporting both E-values, benchmarking against the strength of measured confounders, and interpreting them in context rather than as a pass or fail threshold.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Hours

Steps

  1. Estimate the adjusted effect as a risk ratio, or convert odds ratios, hazard ratios or standardised differences using published approximations.
  2. Compute the E-value for the point estimate and for the confidence limit nearest the null (e.g. with dowhy, the EValue package or the formula).
  3. Benchmark: compare with the associations of the strongest measured confounders with treatment and outcome.
  4. Name plausible unmeasured confounders from qualitative work and judge whether any could reach the E-value.
  5. Report the E-values and benchmarks alongside the estimate, stating the conclusion's robustness in plain language.

Sources

  • causalgraphicalmodels
  • dowhy
  • causalml
  • econml
  • statsmodels
  • VanderWeele, T. J. & Ding, P. (2017) Sensitivity analysis in observational research: introducing the E-value. Annals of Internal Medicine
  • Hernán, M. A. & Robins, J. M. (2020) Causal Inference: What If
  • The Inquiry Series, C5 — Causal Inference in Python — DAGs, Target Trial Emulation & Sensitivity Analysis

Also known as

  • E-value
  • Unmeasured confounding sensitivity analysis

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference