Dirichlet Process Mixture Clustering

Description

Bayesian clustering in which the number of clusters is inferred from the data through a Dirichlet process prior instead of being fixed in advance.

Finite mixtures and latent class models make the analyst fix the number of groups and then compare fits. A Dirichlet process prior instead lets units (users, teams, markets) draw their effects from a discrete distribution with a potentially unlimited number of support points, built by stick-breaking: successive Beta(1, alpha) fractions split the probability among candidate cluster values drawn from a baseline density. The precision alpha governs how readily new clusters form; small values favour few clusters. MCMC then samples cluster allocations and values together, so the output is a posterior distribution over the number of occupied clusters, unit estimates shrunk towards their cluster, and the probability that any two units share a cluster. It is a robust alternative to a single normal random-effects distribution when effects may be lumpy or multimodal.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Choose a baseline density for the unit effects (for example normal on log rates) and decide whether to fix alpha (for example 1 or 5) or give it a gamma prior.
  2. Set a truncation level comfortably above the number of clusters you expect, for example 10 or 20.
  3. Build the stick-breaking weights from Beta(1, alpha) fractions, with the last weight taking the remainder, and draw candidate cluster values from the baseline density.
  4. Sample unit allocations and cluster values by MCMC and check convergence.
  5. Monitor the number of non-empty clusters, and confirm the truncation level is never fully used.
  6. Report the posterior distribution of the number of clusters, pairwise co-clustering probabilities and unit effects, and repeat with a different alpha and baseline to check sensitivity.

Sources

  • Applied Bayesian Modelling (Congdon, 2003)
  • Bayesian Statistical Modelling (Congdon, 2006)

Also known as

  • Dirichlet process mixture
  • Dirichlet process prior clustering
  • Stick-breaking prior
  • Nonparametric Bayesian clustering
  • Infinite mixture model
  • DPP

Categories

  • Analytics & Behaviour
  • Statistics & Modelling