Equivalent-Model Generation and Justification in SEM

Description

Generates the alternative path models that fit the data exactly as well as yours, then argues on design and theory grounds why the chosen model is preferred.

Equivalent models imply the same covariance matrix (or the same d-separation statements) with different paths, so they have identical fit and no amount of data can separate them; near-equivalent models fit almost as well. MacCallum and colleagues found equivalent models in over 80% of published applications, almost never acknowledged, a pervasive form of confirmation bias. Stelzl's and the Lee-Hershberger replacing rules generate equivalents: within a saturated block, a direct effect can be replaced by a reversed effect or a disturbance covariance, and a direct effect between two variables with the same causes can be reversed or replaced. Because these rules can break d-separation equivalence, candidates should be checked in a tool such as dagitty. CFA models have infinitely many equivalents, and a three-variable mediation model has 18. The defence is argument, not fit: temporal order, experimental manipulation, variables that cannot be effects, and design features planned before data collection.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Fit the target model and confirm it is identified and fits acceptably.
  2. Apply the replacing rules (reverse a path, replace it with a disturbance covariance) to generate equivalent and near-equivalent alternatives, starting with the paths most central to the claims.
  3. Check each candidate's d-separation equivalence in a tool such as dagitty and fit near-equivalents to see how close they come.
  4. Discard alternatives contradicted by temporal order, experimental manipulation, fixed attributes such as demographics, or clear implausibility.
  5. Where possible, plan design features before data collection (measurement timing, instruments, manipulated variables) that break the equivalence.
  6. Report the remaining plausible alternatives and the arguments for preferring the retained model; never reverse arrows post hoc simply to pick a winner.

Sources

  • A Beginner's Guide to Structural Equation Modeling (Schumacker & Lomax, 2004)
  • Latent Variable Models (Loehlin & Beaujean, 2017)
  • Principles and Practice of SEM (Kline, 2023)

Also known as

  • Equivalent models check
  • Equivalent models
  • Equivalent models analysis
  • Lee-Hershberger replacing rules

Categories

  • Analytics & Behaviour
  • Statistics & Modelling