Description
Choose each survey item's format by naming its goal first, then its construct, using a decision tree over about fifteen rating, ranking and allocation formats.
Sauro and Lewis classify survey questions as open-ended, closed-ended static (single or multiple choice, rating scales), closed-ended dynamic (adaptive conjoint, MaxDiff, IRT-driven) and task-based. Their catalogue of static rating formats covers linear numeric, Likert agreement, frequency, unipolar and bipolar satisfaction, performance, disconfirmation, comparative, comparative intensity, semantic differential and semantic distance, where a single adjective per row is rated from not at all to perfectly so no polar opposites are needed. Ranking and allocation formats include forced ranking, pick-some (top task) and fixed sum. A decision tree maps goals (discovery, categorical, ranking, allocation, rating, comparison) and constructs to formats, each with documented trade-offs. Format rules follow the evidence: more than three points, a neutral point, positive wording, small grids, counterbalancing. Most format variations shift means by only about 0.3-1.5%, whereas collapsing NPS to three points changed results by 22%.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Active
- Stage
- Design, Evaluation
- Data
- Quantitative
- Effort
- Hours
Steps
- State the goal of each item: discovery, categorical measurement, ranking, allocation, rating or comparison.
- Discovery: use an open-ended item. Categorical: single choice if options are mutually exclusive, multiple choice if not.
- Ranking: forced rank for up to about ten items; pick-some or MaxDiff for more, randomising order. Allocation: fixed sum.
- Rating: choose by construct: linear numeric (general), Likert-type (agreement), frequency, unipolar (none to much), bipolar (opposites), performance (adequacy or quality), disconfirmation or Kano (meeting expectations).
- Comparison: comparative item for two products; comparative intensity against a named benchmark. Adjectives: semantic differential only if true opposites exist, otherwise semantic distance.
- Apply format rules: 5-7 or 11 points, include a neutral point, word positively, keep grids small, counterbalance options and items; record the rationale per item.
Sources
- Survey Design (Sauro & Lewis, 2021)
- Survey and Correlational Research Designs (Privitera, 2014)
- Survey Design (Avedian, 2014)
- Data-Driven Design: Surveys. Tips for Writing Good Questions (Interaction Design Foundation, n.d.)
Also known as
- Survey item decision tree
- 15 common rating scales
- Four classes of survey questions
- Semantic distance scale
- Comparative intensity scale
- Disconfirmation scale
- Survey item-type selection decision tree and rating-scale format catalogue
Categories
- Analytics & Behaviour
- Survey & Psychometrics