Kolmogorov-Smirnov Tests

Description

Compare a sample's empirical cumulative distribution with a hypothesised one, or two samples' distributions with each other, using the largest vertical gap as the test statistic.

The test avoids the arbitrary binning of a chi-square goodness-of-fit test. The empirical distribution function gives the proportion of observations at or below each value and converges uniformly to the true distribution. In the one-sample test, the statistic is the largest absolute difference between the empirical and a fully specified continuous distribution; multiplied by the square root of n, it is compared with the Kolmogorov limiting distribution, rejecting at 5 percent above about 1.36. The two-sample test uses the largest difference between two empirical distributions, scaled by the square root of mn/(m+n), to test whether two independent samples share one continuous distribution. Because it responds to any difference in shape, location or spread, it can show that two designs' task-time distributions differ even when their medians do not. The statistic also yields a confidence band for an unknown distribution.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. State the null: the sample follows a specified continuous distribution (one-sample) or two samples share one distribution (two-sample).
  2. Sort the data and compute the empirical cumulative distribution function(s).
  3. Find the largest absolute vertical difference between the empirical and hypothesised functions, or between the two empirical functions.
  4. Scale it: by the square root of n (one-sample) or of mn/(m+n) (two-sample).
  5. Compare with the Kolmogorov critical value (about 1.36 at 5 percent) or report the p-value.
  6. Plot the empirical distributions together to show where they diverge, and judge whether the difference matters in practice.

Sources

  • Probability and Statistics (DeGroot & Schervish, 2012)
  • Jackknife and Bootstrap, and Goodness-of-Fit using Bootstrap (Babu, n.d.)
  • Mixed Models: Theory and Applications (Demidenko, 2004)
  • Statistical Methods for Survival Data Analysis (Lee & Wang, 2003)
  • Probability Metrics and the Stability of Stochastic Models (Rachev, 1991)

Also known as

  • K-S test
  • KS test
  • Two-sample Kolmogorov-Smirnov test
  • Empirical distribution function test

Categories

  • Analytics & Behaviour
  • Statistics & Modelling