Description
After the tasks, the moderator deliberately takes the opposite position to test how firmly a participant holds a stated view and draw out fuller reasons.
Participants often praise a product they visibly struggled with, out of politeness or self-blame. In this debriefing technique the moderator abandons neutrality, but only after all performance tasks are finished, and challenges the participant's stated view: expressing surprise at the gap between their difficulty and their comments, asking for specific reasons, saying they sense reluctance to criticise, or claiming that other participants felt the opposite. Most participants restate their view with fuller reasons; some admit they were holding back. The point is to reveal how robust an opinion is, not to change it, so a positive view that survives challenge is strong evidence. It is distinct from analyst-side red teaming, which challenges the team's conclusions rather than the participant's. Observers must be warned beforehand, and the moderator backs off if the participant seems intimidated.
Attributes
- Discipline
- Interviewing & Elicitation
- Loss type
- Latent
- Stage
- Evaluation
- Data
- Qualitative
- Effort
- Hours
Steps
- Warn observers and team members beforehand that the moderator will deliberately argue.
- Complete all performance tasks and standard debrief questions first.
- State surprise at the discrepancy between the participant's observed difficulty and their positive comments, and ask for specific reasons.
- Say you sense reluctance to criticise and want to be sure they are not just being kind.
- Optionally claim that other participants felt the opposite, and ask again.
- Back off if the participant seems intimidated, and weight their answers accordingly when analysing.
Sources
- Handbook of Usability Testing (Rubin & Chisnell, 2008)
Also known as
- Devil's advocate technique
- Counter-position challenge in debriefing
Categories
- User Research
- Interviewing & Elicitation