Description
Measure how far one distribution has shifted from another as the area between their cumulative curves, in the units of the measure itself.
Comparing two distributions of task time, load time or satisfaction usually means comparing means or running a Kolmogorov-Smirnov test, whose statistic is the largest vertical gap between cumulative curves and has no units. The Wasserstein distance, for a measure on the real line, is the area between the two cumulative distribution functions. Equivalently it is the average absolute difference between matched quantiles, and the least average distance probability mass must be moved to turn one distribution into the other, which is why it is called the earth mover's distance. Its value is in seconds, points or currency, and reflects how far mass has moved, not just whether the distributions differ. Gini proposed it in 1914 as an index of dissimilarity; Rachev develops its properties and convergence. It needs finite means and, in this source, is descriptive: significance requires a permutation test or bootstrap.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Latent
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Take the two samples to compare (for example task times before and after a redesign).
- Compute each sample's empirical cumulative distribution function.
- Compute the area between the two curves, or equivalently sort both samples, match quantiles and average the absolute differences.
- For a version that emphasises large shifts, average the squared quantile differences and take the square root.
- Interpret the value in the units of the measure (the typical distance mass has moved) and inspect a quantile-quantile plot to see where the shift occurs.
- If a significance statement is needed, use a permutation test or bootstrap interval for the distance.
Sources
- Probability Metrics and the Stability of Stochastic Models (Rachev, 1991)
Also known as
- Wasserstein distance
- Earth mover's distance
- Kantorovich distance
- Kantorovich-Rubinstein metric
- Gini's index of dissimilarity
- Minimal L1 distance
- Kantorovich (Wasserstein, earth mover's) distance between two distributions
Categories
- Analytics & Behaviour
- Statistics & Modelling