Description
Models nested data (events within sessions within users within products) to partition variance and produce correct standard errors for repeated measures.
Multilevel models include random effects for each level of a hierarchy, partitioning variance and accounting for non-independence of observations within units. Without them, repeated observations from one user are treated as independent, inflating effective sample size and producing anti-conservative inference. The treatise calls this essential for any UX study with repeated measures, multiple sessions per user or multi-site data, and notes most UX statistical analysis ignores clustering entirely, treating 100 clicks from one user as 100 observations, which systematically produces false confidence. Mixed models also handle unbalanced designs, crossed random effects (participants and tasks) and growth curves for longitudinal data. Rated HIGH priority.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Identify the nesting or crossing structure (events, sessions, users, tasks, sites).
- Specify fixed effects for predictors of interest and random intercepts (and slopes) for grouping levels.
- Fit with appropriate likelihood (for example glmer for binary outcomes) and check convergence.
- Compute intraclass correlations to see variance at each level.
- Compare models with likelihood ratio tests or information criteria.
- Report fixed effects with correct CIs and variance components.
Sources
- Lost Methods in UX — A Complete Treatise (Gopinath)
- Raudenbush SW & Bryk AS (2002) Hierarchical Linear Models. 2nd ed
- Gelman A & Hill J (2006) Data Analysis Using Regression and Multilevel/Hierarchical Models
- Goldstein H (1986) Multilevel mixed linear model analysis using iterative generalized least squares. Biometrika
Also known as
- Hierarchical linear models
- Mixed-effects models
- Random-effects models
- HLM
- Random effects models
- Partial pooling
- Multilevel (Hierarchical) Models
Categories
- Analytics & Behaviour
- Statistics & Modelling