Description
Naive observers label photographs or video of faces; their agreement shows what faces convey, and comparison with an independently known state shows accuracy.
Instead of coding facial muscle movements, this design asks untrained observers what a face conveys. Agreement studies show faces in random order and collect forced-choice labels from a fixed word list, free labels or pleasant-unpleasant ratings, then compute agreement per face. Accuracy studies record faces under conditions whose emotional state is independently known (a stress versus a calm interview, a surgery film versus a travelogue) and test whether observers can tell them apart. Many early accuracy studies were inconclusive because the criterion emotion was poorly established. Known artefacts matter: happy and sad are default guesses for neutral faces, and omitting a neutral option changes results. For UX, the design validates whether recorded participant faces, avatars, robot faces or emoji-like feedback actually communicate the intended state to ordinary people, and supplies a human benchmark for automated emotion recognition.
Attributes
- Discipline
- Neuro, Biometrics & Affect
- Loss type
- Unmigrated
- Stage
- Design, Evaluation
- Data
- Quantitative
- Effort
- Days
Steps
- Record faces (or render avatars) under conditions whose emotional state is independently known or intended.
- Prepare a response format: forced choice from a word list including a neutral option, free labels, or pleasant-unpleasant ratings.
- Show the stimuli to naive observers in random order, typically 20 or more observers.
- Collect labels or ratings for each face.
- Compute agreement per face and, where a criterion exists, accuracy against the known condition, correcting for chance and default guesses.
- Compare agreement and accuracy across expression sets, designs or automated classifiers.
Sources
- Unmasking the Face (Ekman & Friesen, 2003)
- Physiology of Behavior (Carlson & Birkett, 2017)
- Biological Psychology (Kalat, 2019)
- Biopsychology (Pinel & Barnes, 2022)
- MIT Psychology (Stangor, 2012)
Also known as
- Observer judgement study of facial expressions (accuracy and agreement)
- Emotion recognition judgement study
- Forced-choice facial judgement task
Categories
- Analytics & Behaviour
- Neuro, Biometrics & Affect