Stratified 2x2 Analysis (Cochran-Mantel-Haenszel and Breslow-Day)

Description

Tests and estimates the association between a binary exposure and a binary outcome while holding a stratifying variable constant, and checks whether the association is the same in every stratum.

When a 2x2 association (exposed to a feature or not, succeeded or not) could be confounded or modified by a third variable such as device or tenure, the data are split into one 2x2 table per stratum and analysed with three tests in sequence. The Cochran-Mantel-Haenszel test asks whether exposure and outcome are conditionally independent across strata. The Mantel-Haenszel estimator pools the stratum odds ratios into one adjusted odds ratio with a confidence interval. The Breslow-Day test checks whether the odds ratio is homogeneous across strata; if it is not, the strata show effect modification and stratum-specific odds ratios should be reported instead of the pooled one. This is the epidemiologist's standard guard against Simpson's paradox, where a pooled table shows an association that disappears or reverses within every stratum because the strata differ in size and baseline rate.

Attributes

Loss type
Siloed
Effort
Hours

Steps

  1. Cross-tabulate the binary exposure by the binary outcome separately within each level of the control variable.
  2. Run the Cochran-Mantel-Haenszel test of conditional independence across the strata.
  3. Estimate the Mantel-Haenszel common odds ratio and its 95% confidence interval as the adjusted strength of association.
  4. Run the Breslow-Day test of homogeneity of odds ratios across strata.
  5. If odds ratios are heterogeneous, report stratum-specific odds ratios and treat the stratifier as an effect modifier rather than pooling.
  6. Compare the adjusted odds ratio with the crude (pooled-table) odds ratio to show how much the stratifier changed the picture.

Sources

  • Quantitative Social Research Methods (Singh, 2007)
  • Applied Logistic Regression (Hosmer & Lemeshow, 2000)
  • An Introduction to Categorical Data Analysis (Agresti, 1996)
  • Applied Bayesian Modelling (Congdon, 2003)
  • Statistical Methods for Survival Data Analysis (Lee & Wang, 2003)

Also known as

  • CMH test
  • Mantel-Haenszel test
  • Breslow-Day test
  • Stratified odds ratio
  • Cochran-Mantel-Haenszel test
  • Mantel-Haenszel common odds ratio

Categories

  • Analytics & Behaviour
  • Statistics & Modelling