Words of Estimative Probability

Description

Standardises the probability phrases used in reports (e.g. 'likely' = 55-80%) so readers interpret confidence consistently.

Kent found that analysts and policymakers interpreted words like 'probable' and 'serious possibility' with wildly different probabilities. He proposed a fixed mapping of phrases to probability ranges. Modern intelligence standards (ICD 203, the UK Professional Head of Intelligence Assessment yardstick) require estimative language and separate statements of analytic confidence based on evidence quality. UX research reports are full of 'most users', 'many participants' and 'likely', which stakeholders interpret inconsistently. Adopting a probability yardstick and separate confidence ratings makes research claims calibrated, comparable and auditable.

Attributes

Loss type
Unmigrated
Stage
Definition
Effort
Hours

Steps

  1. Adopt a yardstick, such as remote (<5%), highly unlikely (10-20%), unlikely (25-35%), realistic possibility (40-50%), likely (55-75%), highly likely (80-90%), almost certain (>95%).
  2. Train researchers and stakeholders on the mapping with examples.
  3. For each key finding, state an estimative judgement and a separate confidence level (low, moderate, high) based on evidence quality.
  4. Avoid ambiguous quantifiers ('many', 'some') unless tied to counts or ranges.
  5. Track outcomes of estimative judgements over time and assess calibration.
  6. Include the yardstick in report templates and repositories.

Sources

  • Kent, Words of estimative probability (1964), Studies in Intelligence
  • ODNI, Intelligence Community Directive 203: Analytic Standards (2015)
  • Friedman, War and Chance: Assessing Uncertainty in International Politics (2019)

Also known as

  • WEP
  • Estimative language
  • Kent's words
  • Calibrated verbal probability

Categories

  • Research Operations
  • Research Design & Rigor