ISO 9241-9 Device Assessment Questionnaire

Description

A standard post-task rating questionnaire for pointing and input devices, covering items such as smoothness, force, mental effort, accuracy, speed and overall evaluation.

ISO 9241-9 sets out how to evaluate non-keyboard input devices. Alongside performance testing with Fitts' throughput, it includes a comfort and usability questionnaire answered after each device is used. Items cover aspects such as the force required, smoothness of operation, mental effort, accuracy, speed of cursor movement and an overall evaluation, each on a bipolar rating scale. In Chapter 7 the questionnaire was given after a standard pointing task and again after 15 minutes of in-game play for each of four controllers (mouse, touchpad, Steam controller and DualShock 4), presented in counterbalanced order. Responses were treated as ordinal, compared with Kruskal-Wallis tests and shown as diverging stacked bar charts, and open-ended questions were added because earlier work suggested the standard items alone were not enough. The authors frame the battery as covering three linked concerns, functionality, usability and user experience (McNamara and Kirakowski), so that objective performance, ratings and comments are read together. The questionnaire is a ready-made, comparable way to capture how hardware feels to use.

Attributes

Loss type
Vestigial
Stage
Evaluation
Effort
Days

Steps

  1. Select devices to compare and a counterbalanced order of use.
  2. Have participants complete a standard task (for example multidirectional pointing) and, separately, a realistic in-context task with each device.
  3. Administer the device assessment items after each device.
  4. Add open-ended questions about what was good or bad about each device.
  5. Analyse ratings as ordinal data with non-parametric tests and diverging stacked bar charts.
  6. Read ratings alongside throughput, error data and comments to explain differences.

Sources

  • Garcia-Ruiz, M. (ed.), Games User Research: A Case Study Approach (2016)

Also known as

  • ISO 9241-9 comfort and usability rating
  • ISO 9241-411 device assessment
  • Input device independent questionnaire

Categories

  • Accessibility
  • Human Factors & Error