Graduated Assistance Levels

Description

A four-step ladder of moderator help, from breaking a loop to telling the next step, with rules on when to assist and how to score assisted tasks.

An assist is any intervention that moves a participant towards completion and would not happen without a moderator. Assists let a test continue past an early blocker so later problems can still be observed, but too much help masks them. Dumas and Loring give criteria for assisting: the participant repeats unproductive behaviour, has tried alternatives and asks for help, is heading for a crash, nears the time limit, or wrongly believes the task is done. Think-aloud prompts, encouragement, task clarification and bug recovery are not assists. The ladder has four levels: 1 break the repeating sequence ('what do you think is going on here?'); 2 general hint; 3 specific hint naming the location; 4 tell the next step only. Levels can be skipped. Each task is recorded as completed, completed with assistance (type and count) or not completed, and summative tests restrict assists to level 4.

Attributes

Loss type
Vestigial
Stage
Evaluation
Data
Mixed
Effort
Hours

Steps

  1. Before testing, decide whether assists are allowed, at which levels, and whether assisted completion counts as success.
  2. Wait until it is clear the participant is stuck and why; first ask them to keep trying if options are not exhausted.
  3. Level 1: break the loop with a refocusing question or ask them to reread the task.
  4. Level 2: give a general hint; Level 3: give a specific hint naming the location.
  5. Level 4: tell the next step only; in summative tests use only this level.
  6. Record each task outcome and every assist's level on the data sheet, keeping multiple moderators consistent.
  7. Treat assists as problem evidence and account for them in success rates and ratings.

Sources

  • Usability Testing: A Practitioner's Guide (Hertzum, 2020)
  • Moderated Usability Tests (Dumas & Loring, 2008)
  • Handbook of Usability Testing (Rubin & Chisnell, 2008)
  • Benchmarking the User Experience (Sauro, 2018)

Also known as

  • Graduated assistance levels and assist recording
  • Levels of assistance
  • Hint hierarchy
  • Four-level assist ladder
  • Assist coding
  • Gradual disclosure
  • Graduated prompting

Categories

  • User Research
  • Usability & Evaluation