Description
Nonparametric tests for paired or repeated measures: signs or signed ranks of differences for two conditions, and rank-based Friedman test for three or more.
These tests compare related measurements without assuming normality. The sign test marks each paired difference, or each value against a hypothesised median, as plus or minus and tests the count of pluses against a binomial distribution; it can also estimate the median. The Wilcoxon signed-rank test adds magnitude: take differences, drop zeros, rank absolute differences (averaging ranks for ties), sum the ranks of positive differences and compare with its permutation distribution, or report a standardised Z for larger samples. The Friedman test ranks conditions within each participant and is the nonparametric counterpart of one-way within-subjects ANOVA for three or more conditions. In UX they suit within-subjects comparisons of ordinal ratings or skewed times, which the Mann-Whitney test, built for independent samples, does not cover.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Active
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Hours
Steps
- Confirm the design is paired or repeated (the same participants under each condition) and choose the test: sign test for direction only, Wilcoxon signed-rank for two conditions, Friedman for three or more.
- Compute each participant's difference between conditions, or rank conditions within each participant for Friedman.
- Sign test: count positive differences, drop zeros, and get a p-value from the binomial distribution with p = 0.5.
- Wilcoxon: rank the absolute non-zero differences, average tied ranks, sum the ranks of positive differences, and compare with exact tables or the normal approximation (report Z).
- Friedman: sum ranks per condition and compare the statistic with a chi-square distribution with k - 1 degrees of freedom; follow a significant result with pairwise Wilcoxon tests and a multiple-comparison correction.
- Report the median difference or a confidence interval for it, not just the p-value; ignore ties unless a result narrowly misses significance with several ties.
Sources
- Probability and Statistics (DeGroot & Schervish, 2012)
- Statistics (Witte & Witte, 2017)
- Intermediate Statistics for Dummies (Rumsey, 2007)
- Psychology Statistics for Dummies (Hanna & Dempster, 2012)
Also known as
- Sign test
- Wilcoxon signed-rank test
- Wilcoxon T test for two related samples
- Wilcoxon signed-ranks test
- Signed rank test
- 1-sample Wilcoxon
- Friedman test
- Friedman's rank test for correlated samples
Categories
- Analytics & Behaviour
- Statistics & Modelling