Observer Judgement Study of Facial Expressions

Description

Naive observers label photographs or video of faces; their agreement shows what faces convey, and comparison with an independently known state shows accuracy.

Instead of coding facial muscle movements, this design asks untrained observers what a face conveys. Agreement studies show faces in random order and collect forced-choice labels from a fixed word list, free labels or pleasant-unpleasant ratings, then compute agreement per face. Accuracy studies record faces under conditions whose emotional state is independently known (a stress versus a calm interview, a surgery film versus a travelogue) and test whether observers can tell them apart. Many early accuracy studies were inconclusive because the criterion emotion was poorly established. Known artefacts matter: happy and sad are default guesses for neutral faces, and omitting a neutral option changes results. For UX, the design validates whether recorded participant faces, avatars, robot faces or emoji-like feedback actually communicate the intended state to ordinary people, and supplies a human benchmark for automated emotion recognition.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Record faces (or render avatars) under conditions whose emotional state is independently known or intended.
  2. Prepare a response format: forced choice from a word list including a neutral option, free labels, or pleasant-unpleasant ratings.
  3. Show the stimuli to naive observers in random order, typically 20 or more observers.
  4. Collect labels or ratings for each face.
  5. Compute agreement per face and, where a criterion exists, accuracy against the known condition, correcting for chance and default guesses.
  6. Compare agreement and accuracy across expression sets, designs or automated classifiers.

Sources

  • Unmasking the Face (Ekman & Friesen, 2003)
  • Physiology of Behavior (Carlson & Birkett, 2017)
  • Biological Psychology (Kalat, 2019)
  • Biopsychology (Pinel & Barnes, 2022)
  • MIT Psychology (Stangor, 2012)

Also known as

  • Observer judgement study of facial expressions (accuracy and agreement)
  • Emotion recognition judgement study
  • Forced-choice facial judgement task

Categories

  • Analytics & Behaviour
  • Neuro, Biometrics & Affect