KLM timing with expert user calibration

Description

Predicts expert task times with the Keystroke-Level Model using operator times measured on the actual users and devices, rather than generic 1980s desktop values.

KLM operator values were derived for desktop keyboards and mice in the late 1970s. On touchscreens, specialised consoles, gloves or voice, they can be badly wrong. Expert user calibration measures operator times directly: keystroke or tap time, pointing time (or Fitts parameters), homing, mental preparation and system response, using short benchmark tasks with skilled users. Holleis and colleagues produced mobile operator values in this way. The calibrated model is then used to predict and compare designs. Calibration is the difference between a teaching exercise and a defensible prediction. Rigour comes from calibrating on experts at asymptotic performance, using enough repetitions, and validating predictions on separate tasks.

Attributes

Loss type
Latent
Stage
Design
Effort
Days

Steps

  1. Recruit 5-10 skilled users of the target device or a comparable one.
  2. Design micro-tasks isolating each operator (repeated taps, pointing to targets of varied distance and size, gestures, mental decisions).
  3. Collect at least 20-30 repetitions per operator per user after practice, discarding initial trials.
  4. Compute median operator times and Fitts parameters, and estimate the M operator from tasks with and without decisions.
  5. Build KLM sequences for candidate designs using calibrated values.
  6. Validate on 2-3 full tasks; accept the model if predictions fall within about 20% of observed expert times.

Sources

  • Hollnagel, E. (ed.), Handbook of Cognitive Task Design (2003)
  • Shneiderman, B. & Plaisant, C., Designing the User Interface
  • Gopinath, V., Movie Booking Playbook
  • Handbook of Cognitive Task Design (Hollnagel)
  • Shneiderman & Plaisant
  • Movie Booking Playbook (Gopinath)
  • The Inquiry Series, Ch 16 — Cognitive Task Modeling — GOMS, KLM & CPM-GOMS

Also known as

  • Calibrated Keystroke-Level Model
  • Locally calibrated KLM

Categories

  • User Research
  • Task Analysis & Cognitive Modelling