Description
An interval, mean plus or minus k standard deviations, expected to contain a chosen share of individual values with a chosen confidence, unlike a confidence interval for the mean.
A tolerance interval answers a question UX metrics often imply but rarely ask: within what range do most individual users fall? It is computed as the sample mean plus or minus k times the sample standard deviation, where k comes from tables for the chosen coverage (for example 99 percent of values), confidence (for example 95 percent) and sample size. A one-sided lower bound, mean minus k s, is used when a quantity must stay above a minimum, and an upper bound when it must stay below a maximum. Montgomery and Runger stress the contrast with neighbouring intervals: a confidence interval covers the mean, a prediction interval covers the next single observation, and a tolerance interval covers a stated share of the population. Tolerance intervals are far wider than confidence intervals, especially at small n, and the standard factors assume normally distributed data.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Choose the coverage proportion (for example 90 or 99 percent of individual values) and the confidence level (for example 95 percent).
- Compute the sample mean and standard deviation and note n, transforming skewed measures such as task time (for example by logging) to approximate normality.
- Look up the tolerance factor k for that coverage, confidence and n in a two-sided or one-sided table, or compute it with software.
- Report mean minus k s to mean plus k s, or a one-sided bound, back-transforming if needed.
- Interpret it as the range expected to contain the chosen share of individual users, not the mean.
- For clearly non-normal data use the distribution-free (Wilks) form: the interval between two order statistics covers at least the chosen share of the population with a confidence that does not depend on the distribution, so pick the ranks or n from that binomial-type calculation (for example, the sample minimum to maximum covers 95 percent of the population with 90 percent confidence once n reaches 77) (David & Nagaraja, 2003).
Sources
- Applied Statistics and Probability for Engineers: Student Solutions Manual (Montgomery & Runger, 2002)
- Order Statistics (David & Nagaraja, 2003)
- Statistical Design and Analysis of Experiments (Mason et al., 2003)
- Continuous Multivariate Distributions, Vol. 1 (Kotz et al., 2000)
Also known as
- Tolerance interval
- Tolerance bound
- Normal tolerance limits
Categories
- Analytics & Behaviour
- Statistics & Modelling