Description
Compare a sample's empirical cumulative distribution with a hypothesised one, or two samples' distributions with each other, using the largest vertical gap as the test statistic.
The test avoids the arbitrary binning of a chi-square goodness-of-fit test. The empirical distribution function gives the proportion of observations at or below each value and converges uniformly to the true distribution. In the one-sample test, the statistic is the largest absolute difference between the empirical and a fully specified continuous distribution; multiplied by the square root of n, it is compared with the Kolmogorov limiting distribution, rejecting at 5 percent above about 1.36. The two-sample test uses the largest difference between two empirical distributions, scaled by the square root of mn/(m+n), to test whether two independent samples share one continuous distribution. Because it responds to any difference in shape, location or spread, it can show that two designs' task-time distributions differ even when their medians do not. The statistic also yields a confidence band for an unknown distribution.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- State the null: the sample follows a specified continuous distribution (one-sample) or two samples share one distribution (two-sample).
- Sort the data and compute the empirical cumulative distribution function(s).
- Find the largest absolute vertical difference between the empirical and hypothesised functions, or between the two empirical functions.
- Scale it: by the square root of n (one-sample) or of mn/(m+n) (two-sample).
- Compare with the Kolmogorov critical value (about 1.36 at 5 percent) or report the p-value.
- Plot the empirical distributions together to show where they diverge, and judge whether the difference matters in practice.
Sources
- Probability and Statistics (DeGroot & Schervish, 2012)
- Jackknife and Bootstrap, and Goodness-of-Fit using Bootstrap (Babu, n.d.)
- Mixed Models: Theory and Applications (Demidenko, 2004)
- Statistical Methods for Survival Data Analysis (Lee & Wang, 2003)
- Probability Metrics and the Stability of Stochastic Models (Rachev, 1991)
Also known as
- K-S test
- KS test
- Two-sample Kolmogorov-Smirnov test
- Empirical distribution function test
Categories
- Analytics & Behaviour
- Statistics & Modelling