Runs test and perceived-streak analysis

Description

Tests whether apparent streaks in a sequence of successes, failures or metric values exceed chance, using conditional probabilities, serial correlation and a runs test.

People expect random sequences to alternate more than they do, so they see streaks in chance data: 65% of basketball fans called a random sequence 'streak shooting'. Gilovich, Vallone and Tversky tested shooting records by success rates after one, two or three hits versus misses (51% after hits, 54% after misses), serial correlation of successive outcomes, the Wald-Wolfowitz runs test, and a stationarity test that splits records into blocks of four and compares their distribution with the binomial by chi-square. None showed streaks beyond chance. For a numeric series, each value is coded above or below the median and the number of runs compared with its null distribution (exact tables for small n, a normal approximation above about 20); too few runs indicates positive dependence. Later work adds local-streak tests and separates how often people act after a success from how often they succeed.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Order the outcomes in time and code them as success or failure, or as above or below the median for a numeric series.
  2. Compute success rates after one, two and three successes and after the same number of failures.
  3. Compute the serial correlation of successive outcomes.
  4. Count runs and compare with the null distribution: exact tables for small samples, a standardised normal statistic when n exceeds about 20.
  5. Test stationarity by splitting the sequence into blocks and comparing block results with the binomial expectation by chi-square.
  6. Optionally show stakeholders random sequences to calibrate how readily they perceive streaks.

Sources

  • Heuristics and Biases (Gilovich et al., 2002)
  • Preference, Belief, and Similarity (Tversky, 2004)
  • Statistics for Business and Economics (Newbold et al., 2022)
  • The Analysis of Time Series: An Introduction (Chatfield, 2003)
  • Univariate Discrete Distributions (Johnson et al., 1992)

Also known as

  • Runs and conditional-probability test for perceived streaks
  • Hot hand analysis
  • Wald-Wolfowitz runs test for streaks
  • Streak analysis
  • Runs test for randomness of a time series
  • Wald-Wolfowitz runs test
  • Runs test about the median

Categories

  • Analytics & Behaviour
  • Statistics & Modelling