Repeated-measures and mixed (split-plot) ANOVA

Description

Analyses experiments where participants experience several conditions, alone or crossed with between-group factors, using correct error terms and sphericity-adjusted degrees of freedom.

A repeated-measures ANOVA compares three or more conditions measured on the same participants, treating participants as random. Its F test assumes sphericity (equal variances of all pairwise differences); unadjusted tests can run at an actual alpha of .10-.15 at nominal .05, so degrees of freedom are always corrected, preferably with the Greenhouse-Geisser epsilon, since Huynh-Feldt can be liberal in small samples. A split-plot (mixed) design adds one or more between-group factors, such as age band or a learning-sensitive condition, to within-participant factors, combining the feasibility of between-group comparison with the power of within-participant comparison. The between effect is tested against subjects-within-groups error; the within effect and interaction against the within-by-subjects error with sphericity correction or the multivariate approach. Contrasts and simple effects use separate error terms, and equal covariance matrices across groups are assumed.

Attributes

Loss type
Siloed
Stage
Evaluation
Effort
Days

Steps

  1. Decide for each factor whether it is between groups (participant characteristics, strong learning or carry-over) or within participants (conditions everyone can do).
  2. Form one group per level of each between factor and have every participant complete all within-factor levels, with practice trials and Latin square or Williams counterbalancing of order.
  3. Fit the model and test the between effect against subjects-within-groups error.
  4. Test within effects and interactions with Greenhouse-Geisser adjusted degrees of freedom or the multivariate approach; never report unadjusted F.
  5. Follow up with planned contrasts, simple effects or interaction contrasts using separate error terms.
  6. Report omega squared or other effect sizes with confidence intervals, and check equality of covariance matrices across groups.

Sources

  • Research Methods in HCI (Lazar et al., 2017)
  • Designing Experiments and Analyzing Data (Maxwell et al., 2018)
  • Quantitative Psychological Research (Clark-Carter, 2004)
  • Design and Analysis of Experiments, Vol. 1 (Hinkelmann & Kempthorne, 2008)
  • Statistical Design and Analysis of Experiments (Mason et al., 2003)
  • Methods of Multivariate Analysis (Rencher, 2002)
  • Statistics for Marketing and Consumer Research (Mazzocchi, 2008)
  • Statistics (Witte & Witte, 2017)
  • Psychology Statistics for Dummies (Hanna & Dempster, 2012)

Also known as

  • Split-plot design
  • Mixed between- and within-group factorial design
  • Mixed design
  • Repeated-measures ANOVA with sphericity adjustment
  • Within-subjects ANOVA
  • Greenhouse-Geisser correction
  • Huynh-Feldt correction
  • Mixed ANOVA
  • Between-within design
  • Split-plot ANOVA

Categories

  • Analytics & Behaviour
  • Statistics & Modelling