Sentence-Picture Verification

Description

Participants read a sentence implying an object's shape or orientation, then verify a picture; faster responses to matching pictures show they mentally simulated the described situation.

Participants read or hear a sentence that implies an object's orientation or shape ('He hammered the nail into the floor'; 'The ranger saw the eagle in the sky'), then see a picture and press as fast as possible to say whether the pictured object was mentioned. On critical trials the answer is always yes, yet responses are faster when the picture matches the implied orientation or shape. This shows that readers build a perceptual situation model rather than an abstract representation of the words. Effects are tens of milliseconds, so the design needs filler trials with 'no' answers, counterbalanced pictures and enough trials. For UX, the paradigm can test whether instructions, product descriptions or generated text lead people to picture the intended state (for example the orientation of a device or the state of a toggle), which matters for setup instructions and visual consistency.

Attributes

Loss type
Unmigrated
Effort
Weeks

Steps

  1. Write sentence pairs implying different shapes or orientations of the same object, plus filler sentences.
  2. Prepare pictures that match or mismatch each implied version, and filler pictures requiring a 'no' answer.
  3. Counterbalance so each participant sees each item in only one condition.
  4. Present the sentence, then the picture, and record speed and accuracy of the 'was it mentioned?' response.
  5. Run enough trials and participants to detect differences of tens of milliseconds.
  6. Compare verification times for match versus mismatch pictures with mixed-effects models.

Sources

  • Cognitive Psychology (Goldstein, 2019)

Also known as

  • Sentence-picture verification for situation models
  • Implied orientation picture verification

Categories

  • Content
  • Linguistics, Semiotics & Discourse