Description
Estimating the reliability of a single survey item from three or more panel waves while allowing the underlying true score to change between waves.
Two-wave test-retest correlations confuse unreliability with real change. The quasi-simplex model separates them with three or more waves: each observed answer is a true score plus random error, and each true score depends on the previous wave's true score (a lag-1 Markov process). With three waves the model is identified by assuming equal reliabilities or equal error variances across waves; with four or more, fit can be tested. Waves must be far enough apart that earlier answers are not remembered. The result is a reliability estimate for a single item, which cross-sectional methods such as Cronbach's alpha cannot give. Assumptions matter: change must follow the lag-1 pattern, errors must be uncorrelated, and memory effects inflate estimates.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Unmigrated
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Ask the same item, worded identically, in three or more waves spaced far enough apart to prevent recall of earlier answers.
- Specify a model in which each wave's answer equals a true score plus random error, and each true score is a function of the previous one.
- Identify the model by constraining reliabilities or error variances to be equal across waves.
- Fit it in structural equation modelling software; with four or more waves, test model fit.
- Report the item's reliability and compare items or formats.
Sources
- The SAGE Handbook of Survey Methodology (Wolf et al., 2016)
- Latent Variable Models (Loehlin & Beaujean, 2017)
Also known as
- Quasi-Markov simplex model
- Heise reliability model
- Multi-wave reinterview reliability
Categories
- Analytics & Behaviour
- Survey & Psychometrics