Description
Models item responses as a function of person level and item difficulty to test unidimensionality, differential item functioning and interval properties of UX scales.
IRT models the probability of a response as a joint function of a person's latent trait and item parameters (difficulty, discrimination). The Rasch model, a one-parameter form, gives specific objectivity and interval-level measurement if data fit. The treatise argues UX questionnaires should be validated this way before use: does a 5-item satisfaction scale measure one construct or conflate two? Are some items too easy (everyone agrees) or too hard? Do items function differently for novices and experts (differential item functioning)? IRT answers these; Cronbach's alpha does not. It also enables computerised adaptive testing and linking of scales across versions. Rated HIGH priority (evidence 5, impact 5).
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Siloed
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Weeks
Steps
- Assemble item responses from a sample large enough for the model (200+ for Rasch, more for 2PL).
- Check dimensionality (for example residual PCA, confirmatory factor analysis).
- Fit Rasch or 2PL/graded response models and examine item and person fit statistics.
- Inspect item difficulty against person distribution (Wright map) for targeting gaps.
- Test differential item functioning across key groups (age, language, expertise).
- Revise, drop or add items and re-fit; publish scale properties.
Sources
- Lost Methods in UX — A Complete Treatise (Gopinath)
- Rasch G (1960) Probabilistic Models for Some Intelligence and Attainment Tests
- Bond TG & Fox CM (2015) Applying the Rasch Model. 3rd ed
- Embretson SE & Reise SP (2000) Item Response Theory for Psychologists
- Rasch, Probabilistic Models for Some Intelligence and Attainment Tests (1960)
- Bond & Fox, Applying the Rasch Model (3rd ed., 2015)
- Andrich, A rating formulation for ordered response categories (1978), Psychometrika
- Tenenbaum, G. & Eklund, R. (eds), Handbook of Sport Psychology, 3rd ed. (2007)
Also known as
- IRT
- Rasch measurement
- Latent trait theory
- Rasch model
- One-parameter logistic model
- Rasch analysis
- Item Response Theory
Categories
- Analytics & Behaviour
- Survey & Psychometrics