Sign Test, Wilcoxon Signed-Rank and Friedman Tests

Description

Nonparametric tests for paired or repeated measures: signs or signed ranks of differences for two conditions, and rank-based Friedman test for three or more.

These tests compare related measurements without assuming normality. The sign test marks each paired difference, or each value against a hypothesised median, as plus or minus and tests the count of pluses against a binomial distribution; it can also estimate the median. The Wilcoxon signed-rank test adds magnitude: take differences, drop zeros, rank absolute differences (averaging ranks for ties), sum the ranks of positive differences and compare with its permutation distribution, or report a standardised Z for larger samples. The Friedman test ranks conditions within each participant and is the nonparametric counterpart of one-way within-subjects ANOVA for three or more conditions. In UX they suit within-subjects comparisons of ordinal ratings or skewed times, which the Mann-Whitney test, built for independent samples, does not cover.

Attributes

Loss type
Active
Stage
Evaluation
Effort
Hours

Steps

  1. Confirm the design is paired or repeated (the same participants under each condition) and choose the test: sign test for direction only, Wilcoxon signed-rank for two conditions, Friedman for three or more.
  2. Compute each participant's difference between conditions, or rank conditions within each participant for Friedman.
  3. Sign test: count positive differences, drop zeros, and get a p-value from the binomial distribution with p = 0.5.
  4. Wilcoxon: rank the absolute non-zero differences, average tied ranks, sum the ranks of positive differences, and compare with exact tables or the normal approximation (report Z).
  5. Friedman: sum ranks per condition and compare the statistic with a chi-square distribution with k - 1 degrees of freedom; follow a significant result with pairwise Wilcoxon tests and a multiple-comparison correction.
  6. Report the median difference or a confidence interval for it, not just the p-value; ignore ties unless a result narrowly misses significance with several ties.

Sources

  • Probability and Statistics (DeGroot & Schervish, 2012)
  • Statistics (Witte & Witte, 2017)
  • Intermediate Statistics for Dummies (Rumsey, 2007)
  • Psychology Statistics for Dummies (Hanna & Dempster, 2012)

Also known as

  • Sign test
  • Wilcoxon signed-rank test
  • Wilcoxon T test for two related samples
  • Wilcoxon signed-ranks test
  • Signed rank test
  • 1-sample Wilcoxon
  • Friedman test
  • Friedman's rank test for correlated samples

Categories

  • Analytics & Behaviour
  • Statistics & Modelling