Usability Specification and UX Targets

Description

Specify measurable usability targets with worst, target and best levels before design, then test against them, iterating until targets are met or using them as contractual acceptance criteria.

Instead of deciding after a test which problems matter, the team writes measurable usability requirements up front. Each row of the specification table names a user class and task, an attribute (initial or long-term performance, learnability, retention, satisfaction), a measuring instrument (benchmark tasks, a questionnaire such as SUS), a metric, the current or baseline level, and worst acceptable, target and best attainable levels. Testing then reports observed results against targets, and iteration ends when targets are met, giving an explicit stopping rule. Shneiderman extends this to acceptance testing: for example at least 30 of 35 specified participants complete benchmark tasks within 30 minutes, with separate criteria for older adults, disabled users and retention after a week. Results can also be expressed as ratios to expert performance or target page visits. Targets need team commitment, piloted benchmark tasks and one-sample tests to claim attainment.

Attributes

Loss type
Vestigial
Effort
Weeks

Steps

  1. Choose user classes, key tasks and the usability attributes that matter (effectiveness, efficiency, learnability, retention, satisfaction).
  2. Define a measuring instrument and metric for each, such as benchmark-task time, completion rate, errors or a SUS score.
  3. Measure a baseline on the existing or competing system.
  4. Set worst acceptable, target and best attainable levels with users and developers, for example SUS worst 71, target 80, best 90, keeping about a dozen attainable targets.
  5. For contractual use, specify participant profile, numbers, training and pass thresholds, such as 30 of 35 completing tasks within 30 minutes, plus criteria for special groups and retention.
  6. Pilot benchmark tasks, then test and record observed results against each target, using one-sample tests and confidence intervals to claim attainment.
  7. Redesign and retest until targets are met; express gaps as ratios to expert time or target page visits to locate the worst tasks.

Sources

  • Research-Based Web Design & Usability Guidelines (HHS, 2006)
  • The UX Book (Hartson & Pyla, 2019)
  • Usability Testing: A Practitioner's Guide (Hertzum, 2020)
  • Designing the User Interface (Shneiderman et al., 2018)
  • Measuring the User Experience (Albert & Tullis, 2022)
  • Handbook of Usability Testing (Rubin & Chisnell, 2008)
  • Human-Computer Interaction (Dix et al., 2004)
  • Usability Engineering (Nielsen, 1993)

Also known as

  • Usability specification
  • Usability specification table
  • UX target table
  • UX targets
  • UX goals, metrics and targets
  • Usability goal setting
  • Usability acceptance test
  • Usability acceptance criteria
  • Contractual usability requirements
  • Worst/target/best levels
  • Comparison to goals and expert performance
  • Expert time ratio
  • Page-view efficiency

Categories

  • User Research
  • Usability & Evaluation