Mann-Whitney U test

Description

A non-parametric test comparing two independent groups on ordinal or skewed outcomes, such as task times or rating scores, without assuming normality.

The Mann-Whitney U test ranks all observations from both groups together and tests whether values from one group tend to be larger than those from the other. It is the default alternative to the independent t-test when outcomes are ordinal (single Likert items, SEQ scores) or heavily skewed (task times, error counts) and samples are small. Its null hypothesis is stochastic equality, P(X greater than Y) = 0.5; it only tests medians if both distributions have the same shape. Rigour comes from reporting the U statistic, exact or normal-approximation p-value, tie handling and an effect size such as the rank-biserial correlation or common-language effect size, rather than a p-value alone. For paired designs the Wilcoxon signed-rank test is used instead.

Attributes

Loss type
Active
Stage
Evaluation
Effort
Hours

Steps

  1. Confirm the design has two independent groups and the outcome is ordinal or a skewed continuous measure.
  2. Plot both distributions (histograms or box plots) and check whether shapes are similar before interpreting the result as a median difference.
  3. Run the test (scipy.stats.mannwhitneyu or pingouin.mwu) with a two-sided alternative unless a direction was pre-registered.
  4. Use the exact test for small samples (roughly n below 20 per group without ties) and the normal approximation with tie correction otherwise.
  5. Report U, p, group medians with interquartile ranges, and the rank-biserial correlation (about 0.1 small, 0.3 medium, 0.5 large).
  6. For task times, consider also comparing geometric means on log-transformed data as Sauro and Lewis recommend.

Sources

  • scipy.stats
  • pingouin
  • mord
  • PyMC
  • bambi
  • statsmodels
  • matplotlib
  • seaborn
  • Sauro, J. & Lewis, J. R. (2016) Quantifying the User Experience, 2nd ed.
  • Mann, H. B. & Whitney, D. R. (1947) On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics
  • The Inquiry Series, C2 — Quantitative UX Analysis in Python — Descriptive to Inferential
  • Statistics for Behavioral Sciences
  • Discovering Statistics via SPSS (Field)
  • Survey Methodology (Groves)
  • The Inquiry Series, Ch 42 — Ordinal Logic — Treating Likert & Survey Data Correctly
  • Companion Book to Quantifying the User Experience (Lewis & Sauro)
  • Gratton, C. & Jones, I., Research Methods for Sports Studies (2010)

Also known as

  • Wilcoxon rank-sum test
  • Mann-Whitney-Wilcoxon test
  • Mann-Whitney U test
  • Kruskal-Wallis H test
  • Rank-based non-parametric tests
  • Mann-Whitney & Kruskal-Wallis tests

Categories

  • Analytics & Behaviour
  • Statistics & Modelling