Right-skew detection

Description

Diagnoses whether a UX metric is positively skewed before summarising or testing it, so the right summary statistic, transformation or test is chosen.

Many UX metrics are right-skewed: task times, number of errors, sessions per user, revenue per user, support contacts. Applying means and parametric tests without checking shape produces misleading summaries and inflated error rates. Right-skew detection combines visual and numerical diagnostics: histograms, box plots, Q-Q plots, sample skewness and the comparison of mean and median. With small samples, formal normality tests such as Shapiro-Wilk have low power and with large samples they flag trivial departures, so visual checks and sample skewness are emphasised. The output is a decision about analysis, such as log transform, non-parametric test, bootstrap or modelling a mixture, not a test result in itself.

Attributes

Loss type
Latent
Effort
Hours

Steps

  1. Plot a histogram (with sensible bin widths) and a box plot of the metric, and a Q-Q plot against a normal distribution.
  2. Compute the mean, median, sample skewness and the ratio of mean to median; skewness above about 1, or a mean noticeably above the median, indicates substantial right skew.
  3. Inspect the right tail for outliers and check whether they are genuine (slow users) or errors (abandoned sessions, logging issues).
  4. Check for multiple peaks that suggest mixed populations rather than simple skew.
  5. Choose the analysis: log transform for approximately log-normal data, rank-based tests, bootstrap confidence intervals, or generalised linear models (for example gamma or negative binomial).
  6. Document the diagnostics and the chosen approach in the analysis plan or report.

Sources

  • Quantifying the User Experience (Sauro & Lewis)
  • Statistics for HCI (Dix)
  • Practical Statistics for Data Scientists (Bruce)
  • The Inquiry Series, Ch 39 — The Distribution of Human Performance — Why Averages Lie
  • Myatt, G., Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining (2007)

Also known as

  • Skewness diagnosis
  • Distribution shape check
  • Normality assessment

Categories

  • Analytics & Behaviour
  • Statistics & Modelling