Description
Judge decisions by how they were made: derive a local model of good process from past decisions with good and bad outcomes, then audit consequentialism, structuring and trade-offs.
A good decision can turn out badly and a bad one can get lucky, so judging decisions only by outcomes teaches the wrong lessons. Shaw proposes that a team derive its own model of a good process by comparing many past decisions that ended well or badly and looking for systematic process differences. Following Frisch and Clemen, three features are checked: consequentialism (the choice rested on the likelihood and desirability of outcomes rather than habit or precedent), thorough structuring (options generated, consequences identified, long-term effects not ignored, short-term emotional ones not overweighted) and a compensatory rule (trade-offs were made among consequences rather than one factor deciding). The aim is to make decision rules explicit and to agree in advance an acceptable base rate of error, so that individual bad outcomes do not trigger over-correction.
Attributes
- Discipline
- Game Theory & Decision Science
- Loss type
- Latent
- Stage
- Strategy, Post-launch
- Data
- Qualitative
- Effort
- Weeks
Steps
- Collect a set of past decisions of one kind (launches, research priorities, design bets) with their outcomes, both good and bad.
- Reconstruct each decision's process from memos, tickets and interviews, blind to the outcome where possible.
- Compare processes behind good-outcome and bad-outcome decisions to find systematic differences.
- Score each process for consequentialism, thorough structuring (number of options, consequences, likelihoods) and use of a compensatory trade-off rule.
- Check whether outcomes and stakeholder satisfaction were better when these criteria were met.
- Write the resulting criteria into decision templates, make the decision rule explicit, and agree a shared tolerable error rate.
Sources
- Behavioral Finance and Decision Theory in Investment Management (Wood, 1995)
Also known as
- Decision process quality audit
- Good decision process model
- Frisch-Clemen decision quality criteria
Categories
- Strategy & Collaboration
- Game Theory & Decision Science