Moderation (interaction effects)

Description

Tests whether the effect of one variable on an outcome depends on the level of a third variable, revealing for whom and when a design works.

Moderation asks whether a relationship changes across conditions: does a redesign reduce errors for novices but not experts, or does a trust badge raise conversion only on mobile? It is modelled by adding a product term (X times W) to a regression; a reliable interaction coefficient means the slope of X differs by W. Rigour lies in centring continuous predictors, powering the study for the interaction (which typically needs around four times the sample of a main effect of similar size), probing with simple slopes at meaningful values and Johnson-Neyman regions, and pre-registering moderators rather than hunting through segments. Epidemiology calls the same idea effect modification, and it underpins heterogeneous treatment effect analysis in experimentation.

Attributes

Loss type
Latent
Effort
Days

Steps

  1. Pre-specify the focal predictor, the moderator and the theoretical reason the effect should differ.
  2. Power the study for the interaction specifically; plan roughly four times the sample needed to detect the main effect.
  3. Mean-centre continuous predictors and fit the model with both main effects and the product term.
  4. Probe a significant interaction with simple slopes at the moderator's mean and plus or minus one SD, or at real segment values.
  5. Compute the Johnson-Neyman interval to show the moderator range where the effect is non-zero.
  6. Plot predicted outcomes by moderator level and report the interaction effect size with a confidence interval.

Sources

  • Applied Predictive Modeling (Max Kuhn & Kjell Johnson)
  • R for Marketing Research and Analytics (Chris Chapman & Elea McDonnell Feit)
  • Introduction to Mediation, Moderation, and Conditional Process Analysis (Andrew Hayes)
  • Multiple Regression: Testing and Interpreting Interactions (Leona Aiken & Stephen West)
  • Applied Predictive Modeling (Kuhn & Johnson)
  • R for Marketing Research (Chapman & Feit)
  • The Inquiry Series, Ch 43 — Regression, Moderation & Mediation for UX Researchers

Also known as

  • Moderator analysis
  • Interaction terms
  • Effect modification

Categories

  • Analytics & Behaviour
  • Statistics & Modelling