Documentation Usability Testing

Description

Test documentation either by instructing participants to use it, to judge its quality, or by merely making it available, to see whether and when it is used.

When manuals, tutorials or online help are part of a usability test, the team must first decide which question it is asking, because one session cannot answer both. To learn how well documentation works, participants are instructed to go through the tutorial or use the manual, which yields findings on design, content, organisation and language but is unrealistic. To learn whether people use documentation, participants are given tasks and only told it is available; if few use it, little is learned about revising it. A third option mirrors real delivery, for example omitting a manual few customers would receive and providing a help-desk number, then observing where people get stuck and when they seek help. Findings about the product will always emerge and must be reported, since documentation and software problems are inseparable.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Decide whether the question is the quality of the documentation or whether people use it.
  2. For quality: instruct participants to work through the tutorial or use the manual for set tasks.
  3. For use: give tasks and only mention that documentation is available.
  4. Alternatively mirror real delivery (omit material users would not get; provide help-desk or chat access).
  5. Observe where people get stuck, what they consult and when they seek assistance.
  6. Report findings on both documentation and product, noting which question the set-up answered.

Sources

  • A Practical Guide to Usability Testing (Dumas & Redish, 1999)
  • Usability Engineering (Nielsen, 1993)

Also known as

  • Testing manuals and online help
  • Tutorial testing
  • Help and documentation testing

Categories

  • User Research
  • Usability & Evaluation