Description
Measures whether users can correctly predict what will happen when they act on a label or read a message, as a direct quantitative test of interface language.
Bergen's embodied simulation account suggests understanding language involves mentally simulating what it describes. If interface language is clear, users' simulations of an action's outcome should match what the system does. Predictability testing measures this directly: participants see a label, button or message in context and predict the outcome before acting, choosing from options or describing it freely. Prediction accuracy and confidence quantify clarity, and calibration (confidence versus accuracy) reveals dangerous confident errors. Unlike preference tests or comprehension questions, this targets the action-outcome link that matters for safety. Rigour comes from realistic context, randomised presentation, enough participants per label, and analysing errors by type.
Attributes
- Discipline
- Linguistics, Semiotics & Discourse
- Loss type
- Latent
- Stage
- Design
- Data
- Quantitative
- Effort
- Days
Steps
- Select labels and messages linked to consequential or ambiguous outcomes.
- Present each in realistic context to 15-30 participants (remote unmoderated is suitable) and ask for a predicted outcome and confidence (0-100%).
- Code predictions as correct, partially correct or wrong, ideally with two coders.
- Compute accuracy per label (target at least 80% for routine and 95% for irreversible actions) and identify confident errors (wrong with confidence above 70%).
- Revise low-scoring labels and test alternatives side by side.
- Add the test to content governance for new labels.
Sources
- Fodor, J., The Language of Thought (1975)
- Coulson, S., Semantic Leaps (2001)
- Bergen, B., Louder Than Words (2012)
- Lakoff, G., Women, Fire, and Dangerous Things (1987)
- Language of Thought (Fodor)
- Semantic Leaps (Coulson)
- Louder Than Words (Bergen)
- Women Fire & Dangerous Things (Lakoff)
- The Inquiry Series, Ch 17 — Mentalese, Compositionality & the Language of Thought
Also known as
- Label predictability testing
- Prediction-based content testing
- Outcome prediction test
Categories
- Content
- Linguistics, Semiotics & Discourse