Non-Verbal Cue Analysis in Think-Aloud

Description

Video-code gestures and facial expressions during think-aloud sessions and have multiple interpreters read them, recovering confusion and affect that users never verbalise.

Think-aloud data miss much of what users feel, especially when users fall silent or say something polite. Yammiyavar and colleagues filmed think-aloud sessions with cameras on face and hands as well as screen, replayed them to find recurring and strategically important gestures and expressions, counted their frequency, and extracted stills and clips. These were interpreted by independent reviewers, by acquaintances of the user, by strangers of the same age and culture, and by the user three days later. Gestures quantified, simplified or pointed at fuzzy thoughts; expressions revealed surprise, satisfaction, confusion, deep thought, frustration and boredom, often while the user was silent or saying something contradictory. Triangulating interpreters guards against single-coder bias, and merging agreed interpretations into the verbal protocol produces a richer record of where problems occur.

Attributes

Loss type
Siloed
Stage
Evaluation
Effort
Weeks

Steps

  1. Record think-aloud sessions with a camera on the participant's face and hands as well as screen capture.
  2. Replay recordings to identify recurring and strategically placed gestures and facial expressions.
  3. Count their frequencies per participant and task.
  4. Extract stills and short clips around each cue.
  5. Have at least two independent interpreters, ideally including acquaintances or same-culture strangers, and the participant at a later date, label each cue.
  6. Keep interpretations where interpreters agree and merge them with the verbal protocol, marking where affect contradicts or replaces speech.

Sources

  • Usability and Internationalization (Aykin, 2007)

Also known as

  • Non-verbal cue analysis in think-aloud tests
  • Gesture analysis in think-aloud
  • Facial expression coding in think-aloud

Categories

  • User Research
  • Usability & Evaluation