XR Heuristic Evaluation and Staged XR Usability Engineering

Description

Evaluate AR, VR and MR systems in stages: expert inspection against Nielsen's heuristics plus XR-specific ones, then qualitative and quantitative testing with presence, sickness and workload measures.

Extended reality adds problems that desktop usability methods miss: 3D selection and manipulation, multimodal output, shared physical space, presence and adverse aftereffects. The staged flow runs task analysis, expert inspection, qualitative user testing, then formative and summative quantitative studies. In inspection, three to five experts walk representative tasks and check each step against Nielsen's ten heuristics (77 percent of 672 issues in one VR application mapped to them) extended with XR heuristics: users always know their own location and spatial relations; superimposed objects are registered and persist regardless of head or camera speed; real-world size and conventions are respected; scanning and frame-of-reference transformations are minimised; spatial audio and multimodal cues support attention; there is an explicit exit; controls outside the field of view can be found; virtual clutter is limited; and tracking latency, comfort, sickness, setup and physical-space safety are considered. User testing then adds questionnaires (SUS, Presence Questionnaire, SSQ, NASA-TLX), performance and physiological measures.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Weeks

Steps

  1. Analyse the target XR tasks, interactions and information needs.
  2. Have three to five experts walk each task and check every step against Nielsen's heuristics plus the XR additions: location awareness, registration and persistence, real-world scale, frame-of-reference load, exits, out-of-view controls, clutter, comfort and sickness, setup and physical safety.
  3. Consolidate violations, report them to the design team and redesign.
  4. Run qualitative testing (think-aloud, interviews) to see how users perceive and approach the XR interaction.
  5. Run formative then summative studies with task performance, XR questionnaires for presence, simulator sickness, workload and satisfaction, and where useful implicit physiological measures.
  6. Benchmark against a real-world or non-XR equivalent of the task where possible.

Sources

  • Handbook of Human Factors and Ergonomics (Salvendy & Karwowski, 2021)

Also known as

  • AR/VR heuristic evaluation
  • XR usability engineering
  • Sequential evaluation for virtual environments
  • XR heuristic evaluation

Categories

  • User Research
  • Usability & Evaluation