Description
Analyses task times on a log scale because they are positively skewed, reporting geometric means and back-transformed confidence intervals rather than arithmetic means.
Task completion times are bounded at zero and strongly right-skewed: most users finish near a typical time while a few take much longer. Arithmetic means and normal-theory confidence intervals are therefore biased upward and unstable, particularly in the small samples typical of usability studies. The protocol log-transforms each time, computes means and confidence intervals on the log scale, and back-transforms them, yielding the geometric mean and asymmetric intervals. Research by Sauro and Lewis shows the geometric mean is the better estimate of the median for samples under about 25. Plotting the log-scale distribution also exposes structure, such as bimodality, that indicates distinct user strategies hidden by an average.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Latent
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Hours
Steps
- For any task time data:
- Plot histogram and QQ plot.
- (2) Apply Shapiro-Wilk test (n<50) or Kolmogorov-Smirnov (n≥50).
- (3) If right-skewed: compute ln(time) for each observation.
- (4) Run t-test on log-transformed data.
- (5) Back-transform: geometric mean = exp(mean_log), CI bounds = exp(mean_log ± 1.96×se_log).
- (6) Report: 'Geometric mean task time: 45s [95% CI: 38–53s]'.
Sources
- Measuring the User Experience — Tullis & Albert
- Quantifying the User Experience (Sauro & Lewis)
- Statistics for HCI (Dix)
- Practical Statistics for Data Scientists (Bruce)
- PyMC
- The Inquiry Series, Ch 39 — The Distribution of Human Performance — Why Averages Lie
- scipy.stats
- pingouin
- mord
- bambi
- statsmodels
- matplotlib
- seaborn
- The Inquiry Series, C2 — Quantitative UX Analysis in Python — Descriptive to Inferential
- Companion Book to Quantifying the User Experience (Lewis & Sauro)
- How to Conduct a Quantitative Usability Test (Sauro)
- Benchmarking the User Experience (Sauro)
Also known as
- Geometric mean task time
- Log-normal task time analysis
- Log-transformation of task times
- Log-normal distribution fitting
- Log-scale time analysis
Categories
- Analytics & Behaviour
- Statistics & Modelling