Description
Tests and estimates the association between a binary exposure and a binary outcome while holding a stratifying variable constant, and checks whether the association is the same in every stratum.
When a 2x2 association (exposed to a feature or not, succeeded or not) could be confounded or modified by a third variable such as device or tenure, the data are split into one 2x2 table per stratum and analysed with three tests in sequence. The Cochran-Mantel-Haenszel test asks whether exposure and outcome are conditionally independent across strata. The Mantel-Haenszel estimator pools the stratum odds ratios into one adjusted odds ratio with a confidence interval. The Breslow-Day test checks whether the odds ratio is homogeneous across strata; if it is not, the strata show effect modification and stratum-specific odds ratios should be reported instead of the pooled one. This is the epidemiologist's standard guard against Simpson's paradox, where a pooled table shows an association that disappears or reverses within every stratum because the strata differ in size and baseline rate.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Cross-tabulate the binary exposure by the binary outcome separately within each level of the control variable.
- Run the Cochran-Mantel-Haenszel test of conditional independence across the strata.
- Estimate the Mantel-Haenszel common odds ratio and its 95% confidence interval as the adjusted strength of association.
- Run the Breslow-Day test of homogeneity of odds ratios across strata.
- If odds ratios are heterogeneous, report stratum-specific odds ratios and treat the stratifier as an effect modifier rather than pooling.
- Compare the adjusted odds ratio with the crude (pooled-table) odds ratio to show how much the stratifier changed the picture.
Sources
- Quantitative Social Research Methods (Singh, 2007)
- Applied Logistic Regression (Hosmer & Lemeshow, 2000)
- An Introduction to Categorical Data Analysis (Agresti, 1996)
- Applied Bayesian Modelling (Congdon, 2003)
- Statistical Methods for Survival Data Analysis (Lee & Wang, 2003)
Also known as
- CMH test
- Mantel-Haenszel test
- Breslow-Day test
- Stratified odds ratio
- Cochran-Mantel-Haenszel test
- Mantel-Haenszel common odds ratio
Categories
- Analytics & Behaviour
- Statistics & Modelling