Thin-Slice Judgement

Description

Naive judges rate a few seconds of recorded behaviour on set dimensions, and their ratings are correlated with judgements from long exposure to test whether short excerpts suffice.

The thin-slice method asks how much of a recorded behaviour stream is needed to judge it. In the original study, three 10-second clips were cut from the beginning, middle and end of each of 13 teachers' classes; naive judges rated them on 15 dimensions such as confident, active, warm and competent, and their ratings correlated highly with end-of-semester evaluations by the teachers' own students (confident r = .82, active r = .77). The rigour lies in the criterion: short-excerpt ratings are not trusted until shown to correlate with ratings from long exposure or with outcomes, dimension by dimension. For research operations it offers a way to validate cheaper coding: if raters of 30-second clips agree with full-session coders, analysis can be sampled. It also exposes how quickly first impressions of an agent, voice or presenter form.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Record whole sessions of the behaviour of interest (sessions, calls, demos, agent conversations).
  2. Cut very short clips at fixed points, for example three 10-second clips from the beginning, middle and end.
  3. Have naive judges, blind to the criterion, rate each clip on a defined set of dimensions.
  4. Obtain a criterion from people with long exposure (full-session coders, regular users) or from outcomes.
  5. Correlate the averaged thin-slice ratings with the criterion, dimension by dimension, and check inter-judge agreement.
  6. Adopt thin-slice coding only for dimensions that correlate strongly with the criterion.

Sources

  • MIT Psychology (Stangor, 2012)
  • Physiology of Behavior (Carlson & Birkett, 2017)

Also known as

  • Thin-slice judgement validated against a criterion
  • Thin slices
  • Ambady-Rosenthal thin-slice method
  • Brief-excerpt rating

Categories

  • Research Operations
  • Research Design & Rigor