Cloze Probability Norming

Description

A norming sample completes text fragments with the next word; the share giving each word is its cloze probability, quantifying what people expect next.

Cloze probability is the proportion of people who supply a given word as the continuation of a fragment; the top answer's share defines how constraining the context is. Psycholinguists use it to build and control stimuli: predictable targets are skipped more often and fixated more briefly in eye-tracked reading, and low-cloze continuations produce a larger N400 brain response, for which cloze is the best single predictor. Contexts are commonly split into high and low constraint at around 40%. The same logic extends to music, where listeners sing the expected next note. The rigour lies in a separate norming sample from the test sample and in population-specific norms. For UX, cloze norming offers a quantitative way to measure what label, word or next step users expect in microcopy, menus, autocomplete, search suggestions or voice replies.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Write context fragments ending just before the target word or step, including predictable and unpredictable versions where you are building stimuli.
  2. Give the fragments to a norming sample, each person supplying the first word or option that comes to mind.
  3. Tabulate completions, normalising spelling and obvious synonyms by a stated rule.
  4. Compute each continuation's cloze probability as the proportion of respondents giving it, and each fragment's constraint as the top answer's share.
  5. Classify fragments as high or low constraint (for example at a 40% threshold).
  6. Use the normed items with a different sample for reading, eye-tracking or interface tests, and compare behaviour by predictability.

Sources

  • The Mind (Goldstein, 2020)
  • Semantic Leaps (Coulson, 2001)

Also known as

  • Cloze probability norming of word and note predictability
  • Cloze norming of sentence constraint and word predictability
  • Cloze norming
  • Cloze probability
  • Sentence completion norming

Categories

  • Content
  • Linguistics, Semiotics & Discourse