Reaction time distribution analysis

Description

Analyses the full shape of response-time distributions, not just means, to reveal slow tails, lapses and distinct processes that averages conceal.

Response times are positively skewed: most responses cluster near a mode with a long right tail. Means are pulled by the tail and medians ignore it, yet the tail often carries the safety-relevant information, such as occasional very slow responses to alerts. RT distribution analysis fits models such as the ex-Gaussian (a Gaussian component mu, sigma plus an exponential tail tau) or log-normal, or examines quantiles and cumulative distribution functions across conditions. A design change that shifts mu speeds routine responses, whereas one that reduces tau eliminates lapses. Rigour comes from enough trials per participant (ideally 100+ for ex-Gaussian fits), proper outlier handling decided in advance, and comparing quantiles rather than only means.

Attributes

Loss type
Unmigrated
Stage
Evaluation
Effort
Weeks

Steps

  1. Collect trial-level response times, ideally 50-100+ trials per participant per condition, with timestamps accurate to about 10 ms.
  2. Pre-register outlier rules (for example exclude anticipations under 150 ms) instead of trimming by standard deviations after the fact.
  3. Plot histograms, quantile (vincentised) plots and cumulative distributions per condition.
  4. Fit ex-Gaussian or log-normal models per participant and condition, reporting mu, sigma and tau or log-scale parameters.
  5. Compare conditions at 10th, 50th, 90th and 99th percentiles, and compare tau to detect differences in lapses.
  6. Translate findings into design implications: tail problems suggest attention or alerting issues, central shifts suggest perceptual or motor issues.

Sources

  • Goldstein, E. B., Cognitive Psychology: Connecting Mind, Research and Everyday Experience
  • Card, S., Moran, T. & Newell, A., The Psychology of Human-Computer Interaction (1983) and GOMS/KLM literature
  • Sauro, J. & Lewis, J. R., Quantifying the User Experience (2nd ed., 2016)
  • Cognitive Psychology (Goldstein)
  • GOMS/KLM literature
  • Quantifying the User Experience (Sauro & Lewis)
  • The Inquiry Series, Ch 4 — Time, Rhythm & the Three Timescales of Experience

Also known as

  • RT distribution analysis
  • Ex-Gaussian RT analysis
  • Response time distribution modelling
  • Vincent averaging

Categories

  • Analytics & Behaviour
  • Statistics & Modelling