Post-Release Follow-Up Study of Installed Systems

Description

A planned study of how a released product is actually used, combining field visits, logging, complaints and economic data, treating the release as the prototype of the next version.

After release, Nielsen argues, the main usability objective is data for the next version: the released product is a prototype of its successor. The study is planned as a lifecycle stage rather than left to chance. It combines several sources: routine marketing satisfaction data compared with lab predictions, field visits with interviews, questionnaires and observation of naturally occurring tasks, logging from instrumented versions, passive sources such as complaints, modification requests, help-line calls and defect tracking adapted for usability, course instructors' knowledge of learning problems, and economic data on the quality and cost of users' work compared with pre-introduction baselines. In Nielsen's survey of specialists such studies were rated 4.3 of 5 for impact yet used in only 46% of projects.

Attributes

Loss type
Vestigial
Stage
Post-launch
Data
Mixed
Effort
Months

Steps

  1. Plan the follow-up study before release, with named questions for the next version and a budget.
  2. Collect routine satisfaction and marketing data and compare them with predictions from lab testing.
  3. Visit users to interview, survey and observe actual use on naturally occurring tasks.
  4. Instrument the product to log feature use and errors, with consent and privacy safeguards.
  5. Mine complaints, modification requests, help-line calls and defect reports for usability causes.
  6. Ask trainers and instructors about common learning problems.
  7. Compare economic and work-life data (time, quality, absenteeism) with pre-introduction baselines and feed findings into requirements for the next version.

Sources

  • Usability Engineering (Nielsen, 1993)

Also known as

  • Follow-up studies
  • Field feedback from installed systems
  • Post-installation field study

Categories

  • User Research
  • Field & Ethnographic