Description
Respondents repeatedly choose the most and least important items from small subsets, producing ratio-quality importance scores without scale-use bias.
Rating scales suffer from everyone rating everything as important, and from cultural differences in scale use. MaxDiff shows respondents subsets of items (typically 4-5 per screen) from a list of 10-30, asking for the best and worst. Balanced incomplete block designs ensure each item appears equally and with each other item. Hierarchical Bayes multinomial logit estimates individual-level utilities, which can be rescaled to probability shares. The method discriminates strongly and works across cultures. Vestigial use runs MaxDiff in survey tools with poor designs and only counts-based scores, and uses it for things that are not comparable, such as features with very different scopes.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Vestigial
- Goal
- Prioritise, Measure attitudes
- Stage
- Strategy, Definition
- Data
- Quantitative
- Effort
- Weeks
Steps
- Define a list of comparable items (features, needs, messages), worded at similar levels of specificity.
- Generate a balanced design: each respondent sees each item about 3 times, 4-5 items per set.
- Field to an adequate sample (e.g. 200+ per segment of interest).
- Estimate utilities with hierarchical Bayes MNL; check fit (RLH) and remove speeders.
- Rescale to probability shares and compare segments with credible intervals.
- Optionally add an anchored question to identify absolute importance thresholds.
Sources
- Louviere, Flynn & Marley, Best-Worst Scaling: Theory, Methods and Applications (2015)
- Finn & Louviere, Determining the appropriate response to evidence of public concern (1992), Journal of Public Policy & Marketing
- Factor and Discriminant Analysis (Sauro & Lewis, MeasuringU training)
- Feature Prioritization Methods (slide deck)
Also known as
- MaxDiff
- Best-worst scaling
- BWS
- Maximum difference scaling
Categories
- Analytics & Behaviour
- Survey & Psychometrics