Icon simulation test

Description

Measures whether users correctly interpret icons by showing them in a simulated context and scoring open-ended interpretations against standardised comprehension thresholds.

Standards bodies test public and safety symbols rigorously before adoption: participants see a symbol in a realistic context, describe its meaning in their own words, and judges score the answers against the intended referent. ISO 9186 sets an acceptance criterion of 66.7% correct comprehension for public information symbols, while ANSI Z535.3 requires 85% correct with no more than 5% critical confusions for safety symbols. An icon simulation test brings this protocol to UI: the icon is placed in a mocked-up context screen that simulates the real moment of use, and responses are categorised as correct, partly correct, wrong, opposite or critical confusion. Rigour comes from open-ended rather than multiple-choice responses, independent judging and pre-set thresholds.

Attributes

Loss type
Siloed
Stage
Design
Data
Mixed
Effort
Days

Steps

  1. Define the referent for each icon in one sentence and decide the acceptance threshold: 66.7% correct for general function icons, 85% correct with at most 5% critical confusions for destructive, financial or safety actions.
  2. Build a context screen for each icon that simulates the moment of use without textual labels that give away the meaning.
  3. Recruit at least 50 participants for a standards-grade test, or 20 to 30 per segment for a diagnostic test, representative of the target users' language, age and digital experience.
  4. Show each icon in context in randomised order and ask 'What does this symbol mean?' and 'What would you expect to happen if you selected it?', recording verbatim responses.
  5. Have at least two judges independently categorise each response as correct, partly correct, wrong, opposite or critical confusion, resolving disagreements by discussion and reporting agreement.
  6. Compute per-icon comprehension rates with 95% confidence intervals (adjusted Wald) and compare the lower bound against the threshold for high-stakes icons.
  7. Redesign icons that fail, consider adding text labels, and retest with new participants.

Sources

  • Pocket Universal Principles of Design (Lidwell)
  • Methods Atlas (Gopinath)
  • Embodied Mind (Varela)
  • The Inquiry Series, Ch 20 — Semiotics, Icons & Cross-Cultural Sign Failure

Also known as

  • Icon comprehension test
  • Symbol comprehensibility test
  • ISO 9186 comprehension test
  • Icon intuitiveness test

Categories

  • User Research
  • Cognitive Psychology & Perception