Confidence Intervals for Task Time (Log-Transformed and Median)

Description

Puts honest bounds around how long a task takes, using log times for small samples and the median for large ones.

Task times are positively skewed: most people finish quickly and a few take much longer. The arithmetic mean is pulled upward by the slow tail and a normal-theory interval around it is misleading. For samples under about 25, Sauro and Lewis recommend transforming times to natural logs, computing a t-based interval on the log scale, and exponentiating back, which gives the geometric mean and an asymmetric interval that respects the skew. For larger samples the median is the better centre and its interval comes from binomial order statistics. Both answer the stakeholder question "how long will this take most people?" without one slow participant distorting the picture.

Attributes

Loss type
Siloed
Stage
Evaluation
Effort
Hours

Steps

  1. Decide when the clock starts and stops, and whether failed attempts are excluded (they usually are for time-on-task).
  2. For n under 25: take ln of each time, compute mean and SD of the logs, form the t interval, then exponentiate the mean and both bounds.
  3. For n of 25 or more: sort the times and find the ranks np ± z√(np(1−p)) with p = 0.5; the times at those ranks bound the median.
  4. Report the geometric mean or median with its interval, not the arithmetic mean.
  5. When comparing to a target time, check whether the target lies above the upper bound.

Sources

  • Quantifying the User Experience (Sauro & Lewis)
  • Companion Book to Quantifying the User Experience (Lewis & Sauro)
  • How to Conduct a Quantitative Usability Test (Sauro)
  • Benchmarking the User Experience (Sauro)

Also known as

  • Geometric-mean task time interval
  • Median task time confidence interval

Categories

  • Analytics & Behaviour
  • Statistics & Modelling