Quick uncertainty rules of thumb (square-root rule, rule of three)

Description

Back-of-envelope error bounds: about twice the square root of a count for percentages, and 3/n as the 95% upper bound when an event never occurred in n trials.

Two quick rules give rough uncertainty without software. Dix's square-root rule converts a percentage to a count and takes about twice its square root as an approximate 95% margin; for a large majority apply it to those who did not choose the option, and near 50% use 1.5 times the square root. Dividing by sample size gives a percentage band. Both simplify the binomial variance. The rule of three handles zero events, where the usual interval collapses because its standard error is zero: solving (1 - p)^n = 0.05 gives an upper 95% bound of about 3/n, with 6/n giving roughly 99.75%. Both assume independent trials with a common probability. Neither corrects for biased samples, self-selection or leading wording; as Dix notes, a precise estimate from a biased sample is a good estimate of the wrong thing.

Attributes

Loss type
Latent
Effort
Hours

Steps

  1. Convert the percentage to a count of people (for example 10% of 1,000 is 100).
  2. For a small proportion take the square root of that count and double it; for a large majority do the same on the count who did not choose the option; near 50% multiply the square root by 1.5.
  3. Divide the resulting count margin by the sample size to give an approximate 95% band in percentage points.
  4. For zero observed events in n independent trials, set the lower limit at 0 and the upper 95% limit at 3/n (about 6/n for 99.75%).
  5. Compare the bound with the decision threshold, such as an acceptable error or defect rate.
  6. Before trusting either rule, check independence, sample fairness, self-selection and question wording.

Sources

  • Statistics for HCI (Dix, 2020)
  • Statistics for Business: Decision Making and Analysis (Stine & Foster, 2018)
  • Statistical Methods for Reliability Data (Meeker & Escobar, 1998)
  • Statistics for Dummies (Rumsey, 2003)

Also known as

  • Square-root rule of thumb for count and survey data
  • 2 x root rule
  • Quick and dirty variability estimate for proportions
  • Rule of Three for zero observed events
  • Rule of 3
  • Zero-numerator confidence bound
  • Upper bound for zero events
  • 3/n rule

Categories

  • Analytics & Behaviour
  • Statistics & Modelling