Description
Compares general and last-episode ratings of a product, and orders questions so attention is not drawn to the product before experience is rated.
Asked how much pleasure they got from driving their car, students' answers correlated with the car's market value; asked how much they enjoyed their last drive, there was no correlation. A general question makes people think about the object and its attributes, whereas the actual experience is spent attending to traffic and passengers. Dolan generalises this as the focusing effect: something matters in a judgement because attention is on it while answering. The safeguard is to ask about experience or wellbeing first and collect product information afterwards. Surveys can also contaminate later surveys: Chelsea fans asked before the 2008 Champions League final to predict their happiness were less happy after losing than fans surveyed only afterwards, because the first survey had focused them on the match. For UX, the check compares how general and last-episode ratings relate to an attribute such as plan tier or price.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Unmigrated
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Choose the product, feature or attribute whose experiential value is in question (price paid, plan tier, a premium feature).
- Ask a general rating ('How much do you enjoy using X?') and a last-episode rating ('How much did you enjoy the last time you used X?').
- Measure the attribute independently from records rather than by self-report.
- Compare the correlation of each rating with the attribute; a link only in the general rating signals a focusing effect.
- In satisfaction or wellbeing surveys, place experience questions before any question that mentions the product or attribute.
- Where people are surveyed repeatedly, include a group surveyed only at the second point to detect contamination by the first survey.
Sources
- Happiness by Design (Dolan, 2014)
Also known as
- Focusing-Effect Check (general versus last-episode ratings)
- Focusing illusion check
- Last-episode rating
- Wellbeing-first question ordering
- Repeated focusing effect check
Categories
- Analytics & Behaviour
- Survey & Psychometrics