Analysis of Covariance (ANCOVA)

Description

Compare group means while adjusting for a pre-measured covariate such as a baseline score, increasing precision and reducing bias, after checking that slopes are parallel.

ANCOVA combines ANOVA and regression: group effects are estimated after statistically controlling a continuous covariate that predicts the outcome, usually a pretest or baseline. In randomised designs this shrinks error variance and raises power, sometimes turning a non-significant ANOVA significant. Assumptions are linearity, equal slopes across groups, a covariate measured before treatment and unaffected by it, and constant error variance; measurement error in the covariate biases non-randomised comparisons. Covariates should be chosen at design time from prior evidence, not by screening current data. When slopes differ, the heterogeneity-of-regression model tests the treatment effect at chosen covariate values and Johnson-Neyman regions show where it is significant. Lord's paradox shows gain scores and ANCOVA answer different questions in intact groups. Online experimentation's CUPED is the same regression adjustment under a new name.

Attributes

Loss type
Renamed
Effort
Hours

Steps

  1. Choose covariates at design time from prior evidence (often a pre-period or pretest measure) and measure them before treatment.
  2. Run the plain ANOVA, then fit the ANCOVA with the covariate and request adjusted (marginal) means.
  3. Test homogeneity of slopes by temporarily adding a group-by-covariate interaction; if it is clearly significant, use a separate-slopes model.
  4. With unequal slopes, estimate treatment effects at chosen covariate values and compute Johnson-Neyman regions of significance.
  5. Check linearity and residual variance, then run post hoc comparisons on adjusted means.
  6. Report F tests for covariate and group with effect sizes (such as omega squared); in non-randomised studies, treat adjustment as imperfect and consider whether a gain-score question is the real one.

Sources

  • Statistical Analysis in JASP (Goss-Sampson, 2020)
  • Statistics for Business: Decision Making and Analysis (Stine & Foster, 2018)
  • Designing Experiments and Analyzing Data (Maxwell et al., 2018)
  • Design and Analysis of Experiments, Vol. 1 (Hinkelmann & Kempthorne, 2008)
  • Advanced Quantitative Data Analysis (Cramer, 2003)
  • Statistical Design and Analysis of Experiments (Mason et al., 2003)
  • Linear Statistical Inference and Its Applications (Rao, 1973)

Also known as

  • ANCOVA
  • Analysis of covariance
  • Covariate adjustment
  • Pretest adjustment
  • Heterogeneity of regression (ANCOHET)
  • Johnson-Neyman technique
  • Regions of significance
  • Gain scores versus ANCOVA (Lord's paradox)
  • Change-score analysis

Categories

  • Analytics & Behaviour
  • Statistics & Modelling