Credibility Weighting of Small-Group Estimates (Buhlmann and Buhlmann-Straub)

Description

Blend each small group's own average with the overall average, weighting it more as its data grow and as real differences between groups prove larger.

A small segment, team or store shows a metric well above average: how much of that is real? Credibility theory, developed for insurance pricing, gives a closed-form answer. Buhlmann's estimate is Z times the group mean plus (1 - Z) times the overall mean, with Z = n/(n + k), where n is the group's data size and k is the expected within-group variance divided by the variance between true group means. The quantities are estimated from several groups with analysis-of-variance mean squares; if the between-group variance estimate is negative, Z is zero. Buhlmann-Straub weights by exposure when groups have different sizes. The estimate is the best linear approximation to the Bayesian answer, and closely related to partial pooling in multilevel models. The older limited-fluctuation rule gives full credibility above a size threshold and the square-root rule below it, which the book criticises as arbitrary.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Collect, for each of r groups, the observations (and exposures, such as sessions or users, if group sizes differ).
  2. Estimate the overall mean, exposure-weighted where needed.
  3. Estimate the expected within-group variance as the pooled within-group variance.
  4. Estimate the between-group variance as the variance of group means minus the within-group variance divided by group size (exposure-adjusted in Buhlmann-Straub); set it and Z to zero if negative.
  5. Compute k as within over between variance and each group's Z = n/(n + k) (or exposure/(exposure + k)).
  6. Report each group's credibility estimate, Z times group mean plus (1 - Z) times overall mean, optionally rebalanced so totals match.
  7. Use the shrunken estimates for ranking, targeting or reporting instead of raw group means.

Sources

  • Loss Models: From Data to Decisions (Klugman et al., 2012)

Also known as

  • Credibility theory
  • Buhlmann credibility
  • Buhlmann-Straub credibility
  • Greatest accuracy credibility
  • Limited fluctuation credibility
  • Experience rating
  • Credibility weighting of a small group's estimate: Buhlmann and Buhlmann-Straub with empirical Bayes estimation, and limited fluctuation credibility

Categories

  • Analytics & Behaviour
  • Statistics & Modelling