Stereotype Threat Framing and Demographic-Item Placement

Description

A study-design check that avoids framing tasks as tests of ability and places demographic questions after performance tasks, so that stereotype threat does not depress some participants' results.

Performance can fall when a negative stereotype about one's group becomes relevant. Steele and Aronson gave identical verbal items described either as diagnostic of ability or as a problem-solving exercise; Black students did worse only under the diagnostic framing, and asking students to state their race before a test also lowered their scores. Similar effects appear for women in mathematics, older adults with technology, low-income children and White men told they would be compared with Asian men. Causes include vigilance, thought suppression and anxiety. For UX studies the implication is concrete: usability tasks described as tests, and demographic questions placed first, can depress performance for groups facing stereotypes such as older adults with technology, and so distort comparisons between groups. The check fixes wording and order, and where it matters treats framing as an experimental factor.

Attributes

Loss type
Latent
Stage
Evaluation
Effort
Hours

Steps

  1. Identify groups in the sample that a relevant ability stereotype targets (for example older adults and technology, women and mathematics).
  2. Describe tasks as tests of the product, not the participant, avoiding words such as 'test of your ability'.
  3. Place demographic and self-rated expertise questions after performance tasks.
  4. Where group comparisons matter, manipulate framing or item order as a factor and compare performance by group.
  5. Consider mitigations such as brief affirmation or explaining that the study evaluates the design.
  6. Report framing and item order with the results.

Sources

  • MIT Psychology (Stangor, 2012)

Also known as

  • Stereotype threat check
  • Diagnostic versus non-diagnostic task framing
  • Demographics-last ordering

Categories

  • Research Operations
  • Research Design & Rigor