Natural frequencies and transparent risk formats

Description

Express risks as counts in an imagined population and absolute rather than relative changes, then test whether people's probability judgements become accurate.

People given a test's accuracy and a disease's base rate as percentages routinely neglect the base rate: told a test is 95 percent accurate, many conclude a positive result means a 95 percent chance of disease. Recasting the same information as natural frequencies (of 100,000 people, 100 have the disease, 95 of them test positive, and about 4,995 of the healthy also test positive) makes the correct answer, under 2 percent, easy to read off. The same programme recommends absolute over relative risk (a pill scare reported as a '100 per cent' increase was 1 in 7,000 to 2 in 7,000) and fast-and-frugal decision trees as transparent decision aids. As a research method, formats are compared in experiments on judgement accuracy, and the natural-frequency table is a working tool for checking calculations.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Identify the probability judgement users must make (for example the chance a positive alert is real).
  2. Build a natural-frequency version: pick a reference population (for example 1,000 or 100,000), apply the base rate, then apply sensitivity and false-positive rate to each group.
  3. Lay out a 2 x 2 table of outcome by test result and read the positive predictive value from the positive row.
  4. Create matched presentation formats: conditional probabilities, natural frequencies, icon arrays, and absolute versus relative risk.
  5. Randomly assign participants to formats and ask for the probability judgement and a decision.
  6. Score accuracy against the correct value and compare formats; check comprehension with a teach-back question.

Sources

  • Heuristics and Biases: The Science of Decision Making (Dale, 2015)
  • Cognitive Psychology (Sternberg & Sternberg, 2012)
  • Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102
  • A Practical Guide to Designing with Data (Suda, 2010)

Also known as

  • Natural-frequency calculation for base-rate neglect
  • Base-rate neglect correction
  • Medical test problem solution
  • Natural frequency table
  • Fast-and-frugal trees and transparent risk formats
  • Frequency versus probability risk format

Categories

  • Analytics & Behaviour
  • Behavioral Science & Economics