Description
Elicit the same preference with formally equivalent procedures (choice, matching, pricing, rating, choosing versus rejecting) and compare the orderings to see whether the response mode drives the answer.
Normative theory says preference should not depend on how it is asked. This check runs two or more equivalent procedures in separate groups with the same options. In choice versus matching, one group chooses and other groups each fill a blank so the options are equal; the more important attribute wins far more often in choice (68 versus 4 percent in one safety-versus-cost example). In preference reversals, people choose the safer bet but price the long shot higher; adding a sure amount between the two prices separates intransitivity from overpricing. In choose versus reject, an enriched option with strong positives and negatives can be both chosen and rejected by majorities. Discrepancies show that the instrument is shaping the answer, so no single procedure gives the true preference.
Attributes
- Discipline
- Behavioral Science & Economics
- Loss type
- Latent
- Stage
- Definition, Design
- Data
- Quantitative
- Effort
- Weeks
Steps
- Choose two or more procedures that should yield the same order (choice, matching, pricing, rating, choose versus reject).
- Run them in separate groups with identical options, to avoid memory effects.
- For matching, use four groups that each fill one blank cell, then derive the implied preference.
- Translate every response into an implied preference ordering.
- Compare shares between procedures (for example choice share against matching share) and test the difference.
- For reversals, add a sure amount between the two prices to locate the source of the inconsistency.
- Report which mode was used and the size and direction of discrepancies.
Sources
- Preference, Belief, and Similarity (Tversky, 2004)
- Tversky, A., Sattath, S., & Slovic, P. (1988). Contingent weighting in judgment and choice. Psychological Review, 95
- Heuristics and Biases (Gilovich et al., 2002)
- Judgment under Uncertainty: Heuristics and Biases (1974 paper) (Tversky & Kahneman, 1974)
- Decision Making: Descriptive, Normative, and Prescriptive Interactions (Bell et al., 1988)
Also known as
- Procedure invariance check
- Choice versus rating versus pricing comparison
- Choice versus matching test
- Preference reversal check
- Fractile versus exceedance-odds comparison
Categories
- Analytics & Behaviour
- Behavioral Science & Economics