Anchoring check for survey questions

Description

Audits and counterbalances numbers shown before numeric answers, such as example values, default slider positions and starting price brackets, because they anchor responses.

In the classic paradigm a random number from a wheel of fortune preceded a higher-or-lower question and an estimate; median estimates of the share of African countries in the UN were 25 and 45 for anchors of 10 and 65, and paying for accuracy did not help. Anchors act even when extreme, unattended or known to be irrelevant, and have most effect when expressed on the response scale and dimension. Survey instruments create anchors routinely: example values in placeholders, default slider positions, the first bracket in an unfolding sequence, prices in willingness-to-pay ladders, or earlier numeric questions. Warnings and incentives do not remove anchoring; prompting people to consider the opposite reduces it. The check audits instruments for such numbers, removes or randomises them, or deliberately splits samples across high and low starting points to measure sensitivity.

Attributes

Loss type
Latent
Effort
Days

Steps

  1. Audit questionnaires, pricing questions and moderator scripts for any number that precedes a numeric judgement: examples, defaults, slider start points, first brackets.
  2. Remove anchors that serve no purpose, or replace defaults with blank or neutral starting states.
  3. Where an anchor is unavoidable, split the sample and counterbalance high and low starting values across versions.
  4. Compare median estimates between versions to quantify anchor sensitivity.
  5. Report the difference alongside results, and add a consider-the-opposite prompt where anchors must remain.

Sources

  • Heuristics and Biases (Gilovich et al., 2002)
  • Judgment under Uncertainty: Heuristics and Biases (1974 paper) (Tversky & Kahneman, 1974)
  • On Second Thought: Outsmarting Your Mind's Hard-Wired Habits (Herbert, 2010)

Also known as

  • Anchoring check for numeric and bracketed survey questions
  • Anchor audit
  • Starting-point bias check
  • Wheel-of-fortune anchor paradigm
  • Higher/lower then estimate anchoring test

Categories

  • Analytics & Behaviour
  • Survey & Psychometrics