IHCLab Usability Test for Serious Games

Description

A summative lab method for serious games in which a group of 5 to 20 learners is introduced, shown a live demo, given a task list and then answers selectable questionnaires on heuristics, game experience and use for learning.

The IHCLab method evaluates finished serious games with all participants at once in a lab configured with a computer per user, cameras on faces, observers and a moderator with a projector. Sessions last an hour to an hour and a half in four phases. First, a ten-minute introduction and a short profile questionnaire. Second, a projected live demo that puts the game's controls in context. Third, participants work through a task list in the game while observers watch. Fourth, the team chooses one or more of three questionnaires: a game heuristics questionnaire of ten player-answered questions generalised from Desurvire's playability heuristics (consistency, customisation, predictable controls, skippable non-playable content, status information, help and training); a game experience questionnaire with four 1 to 10 ratings (fun, difficulty adapting to controls, excitement, ease of meeting the objective) and seven 5-point indicators such as efficiency, immersion and learning curve; and a two-question use for learning survey on comfort and motivation to use such a game for education. The method is practical for schools and universities, though its questionnaires are home-grown rather than validated.

Attributes

Loss type
Latent
Stage
Evaluation
Data
Mixed
Effort
Days

Steps

  1. Set up a lab with one machine per participant, face cameras, observer positions and a projector.
  2. Recruit 5 to 20 representative learners and open with an introduction and profile questionnaire.
  3. Give a live projected demo of the game and its controls.
  4. Hand out a task list and let participants complete it while observers note difficulties.
  5. Administer the chosen questionnaires: heuristics, game experience, use for learning, or a combination.
  6. Chart results per item and highlight heuristics or indicators with poor ratings for redesign.

Sources

  • Garcia-Ruiz, M. (ed.), Games User Research: A Case Study Approach (2016)

Also known as

  • Group usability test for serious games
  • IHCLab serious game evaluation methodology

Categories

  • User Research
  • Usability & Evaluation