Sequential Regime-Shift Judgement Task

Description

Participants watch a noisy process one observation at a time and judge after each whether it still works or has shifted, under a payoff scheme, revealing over-reaction and feedback effects.

Many monitoring decisions require telling a real shift from noise. In Shaw's widget-machine task, quality readings from a simulated process appear one at a time; while working, readings stay within a known band with occasional excursions, and at an unknown point the process may break and stay shifted. After each reading the participant judges working or broken. Correct judgements earn a point, wrong ones lose a point, and leaving a broken machine running accumulates losses. Across many participants and 500 trials each, people shut machines down far too often, perceived gains and losses relative to recent reference points, did worse when told after each trial whether they were right, and over-reacted to short trends. Shaw drew the lesson that a fixed pre-set policy outperforms reacting to short-term feedback.

Attributes

Loss type
Unmigrated
Effort
Weeks

Steps

  1. Define the in-control band from past data and show participants a history of in-control readings.
  2. Generate sequences in which the process may shift high or low at an unknown point and persist; include short chance trends in working periods.
  3. Reveal observations one at a time and ask for a working or broken judgement after each.
  4. Score with a payoff matrix that rewards correct judgements, penalises errors and accumulates loss while a broken process runs.
  5. Run many participants over many trials, with and without outcome feedback, and compute the optimal policy for comparison.
  6. Compare false alarms, misses, detection delay and judgement ratios across trend and feedback conditions.

Sources

  • Behavioral Finance and Decision Theory in Investment Management (Wood, 1995)

Also known as

  • Widget machine task
  • Process-change detection judgement task
  • Working versus broken process monitoring exercise

Categories

  • Analytics & Behaviour
  • Behavioral Science & Economics