Statistical Tolerance Intervals

Description

An interval, mean plus or minus k standard deviations, expected to contain a chosen share of individual values with a chosen confidence, unlike a confidence interval for the mean.

A tolerance interval answers a question UX metrics often imply but rarely ask: within what range do most individual users fall? It is computed as the sample mean plus or minus k times the sample standard deviation, where k comes from tables for the chosen coverage (for example 99 percent of values), confidence (for example 95 percent) and sample size. A one-sided lower bound, mean minus k s, is used when a quantity must stay above a minimum, and an upper bound when it must stay below a maximum. Montgomery and Runger stress the contrast with neighbouring intervals: a confidence interval covers the mean, a prediction interval covers the next single observation, and a tolerance interval covers a stated share of the population. Tolerance intervals are far wider than confidence intervals, especially at small n, and the standard factors assume normally distributed data.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Choose the coverage proportion (for example 90 or 99 percent of individual values) and the confidence level (for example 95 percent).
  2. Compute the sample mean and standard deviation and note n, transforming skewed measures such as task time (for example by logging) to approximate normality.
  3. Look up the tolerance factor k for that coverage, confidence and n in a two-sided or one-sided table, or compute it with software.
  4. Report mean minus k s to mean plus k s, or a one-sided bound, back-transforming if needed.
  5. Interpret it as the range expected to contain the chosen share of individual users, not the mean.
  6. For clearly non-normal data use the distribution-free (Wilks) form: the interval between two order statistics covers at least the chosen share of the population with a confidence that does not depend on the distribution, so pick the ranks or n from that binomial-type calculation (for example, the sample minimum to maximum covers 95 percent of the population with 90 percent confidence once n reaches 77) (David & Nagaraja, 2003).

Sources

  • Applied Statistics and Probability for Engineers: Student Solutions Manual (Montgomery & Runger, 2002)
  • Order Statistics (David & Nagaraja, 2003)
  • Statistical Design and Analysis of Experiments (Mason et al., 2003)
  • Continuous Multivariate Distributions, Vol. 1 (Kotz et al., 2000)

Also known as

  • Tolerance interval
  • Tolerance bound
  • Normal tolerance limits

Categories

  • Analytics & Behaviour
  • Statistics & Modelling