Description
Compare group means while adjusting for a pre-measured covariate such as a baseline score, increasing precision and reducing bias, after checking that slopes are parallel.
ANCOVA combines ANOVA and regression: group effects are estimated after statistically controlling a continuous covariate that predicts the outcome, usually a pretest or baseline. In randomised designs this shrinks error variance and raises power, sometimes turning a non-significant ANOVA significant. Assumptions are linearity, equal slopes across groups, a covariate measured before treatment and unaffected by it, and constant error variance; measurement error in the covariate biases non-randomised comparisons. Covariates should be chosen at design time from prior evidence, not by screening current data. When slopes differ, the heterogeneity-of-regression model tests the treatment effect at chosen covariate values and Johnson-Neyman regions show where it is significant. Lord's paradox shows gain scores and ANCOVA answer different questions in intact groups. Online experimentation's CUPED is the same regression adjustment under a new name.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Renamed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Choose covariates at design time from prior evidence (often a pre-period or pretest measure) and measure them before treatment.
- Run the plain ANOVA, then fit the ANCOVA with the covariate and request adjusted (marginal) means.
- Test homogeneity of slopes by temporarily adding a group-by-covariate interaction; if it is clearly significant, use a separate-slopes model.
- With unequal slopes, estimate treatment effects at chosen covariate values and compute Johnson-Neyman regions of significance.
- Check linearity and residual variance, then run post hoc comparisons on adjusted means.
- Report F tests for covariate and group with effect sizes (such as omega squared); in non-randomised studies, treat adjustment as imperfect and consider whether a gain-score question is the real one.
Sources
- Statistical Analysis in JASP (Goss-Sampson, 2020)
- Statistics for Business: Decision Making and Analysis (Stine & Foster, 2018)
- Designing Experiments and Analyzing Data (Maxwell et al., 2018)
- Design and Analysis of Experiments, Vol. 1 (Hinkelmann & Kempthorne, 2008)
- Advanced Quantitative Data Analysis (Cramer, 2003)
- Statistical Design and Analysis of Experiments (Mason et al., 2003)
- Linear Statistical Inference and Its Applications (Rao, 1973)
Also known as
- ANCOVA
- Analysis of covariance
- Covariate adjustment
- Pretest adjustment
- Heterogeneity of regression (ANCOHET)
- Johnson-Neyman technique
- Regions of significance
- Gain scores versus ANCOVA (Lord's paradox)
- Change-score analysis
Categories
- Analytics & Behaviour
- Statistics & Modelling