Staircase elicitation of upper and lower probability

Description

Respondents mark each value on a probability ladder as too low, about right or too high; the boundaries give lower and upper probabilities, exposing how vague a belief is.

Point probability estimates hide how uncertain people are about their own judgement. In the staircase procedure, respondents see probability values from 0 to 1 in steps and place each in a verbal category from 'clearly too low' to 'clearly too high'. The lowest value not marked clearly too low is the lower probability and the highest not marked clearly too high is the upper probability; the interval width measures vagueness without asking for probabilities of probabilities. Tversky and Koehler used it with football fans to test whether interval judgements behave as lower-probability theory requires. Lower and upper judgements for complementary events were complementary (summing to about 1.02), but they violated additivity in the way support theory predicts, so intervals are subadditive just like point judgements. Category wording affects the bounds and long ladders tire respondents.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. State the event precisely and prepare a ladder of probability values in steps (for example every .05).
  2. For each value ask the respondent to tick one category: clearly too low, probably too low, about right, probably too high, clearly too high.
  3. Take the lowest value not marked clearly too low as the lower bound and the highest not marked clearly too high as the upper bound.
  4. Ask the same for the complementary event and check that lower and upper bounds are roughly complementary (sum near 1).
  5. Report interval widths per respondent and event as a measure of vagueness, alongside any point estimate.

Sources

  • Preference, Belief, and Similarity (Tversky, 2004)
  • Operational Subjective Statistical Methods (Lad, 1996)

Also known as

  • Vagueness staircase
  • Interval probability elicitation
  • Upper and lower probability elicitation

Categories

  • Analytics & Behaviour
  • Survey & Psychometrics