Individualised mixture profiles

Description

Describe each user as their own weighting of shared behaviour components, shrunk towards the population mix, and score models by out-of-sample predictive bits per item.

The method applies latent-variable thinking to transaction or usage data. A global set of K mixture components describes typical baskets or sessions, fitted with EM on all transactions. Each individual then gets weights over those components estimated from their own history, with a Dirichlet prior that pulls people with little data towards the population weights. Each person's history is split by time, 70 per cent for training and 30 per cent for testing, and models are scored as predictive entropy in bits per item; a random guess scores log2 of the number of items. Components are described by items with lift above 1. On drugstore data, individualised models beat the global model across 1-200 components. The same chapter fits a two-type mixture of shopping rhythms (random versus regular inter-purchase gaps). Unlike hard segmentation, users are not forced into one segment.

Attributes

Loss type
Unmigrated
Effort
Weeks

Steps

  1. Choose the number of components K, trying a range.
  2. Fit global components with EM on all transactions or sessions.
  3. Estimate each person's component weights with Dirichlet shrinkage towards the population mix.
  4. Split each person's history by time, 70 per cent train and 30 per cent test.
  5. Score out-of-sample bits per item against the random baseline and the global model, and select K.
  6. Describe each component by its high-lift items and interpret individual profiles as mixtures.

Sources

  • Handbook of Data Mining (Ye, 2003)

Also known as

  • Individualised mixture profiles with bits-per-item scoring
  • Individual-level mixture model
  • Personalised mixture of behaviours
  • Bits per item

Categories

  • Analytics & Behaviour
  • Statistics & Modelling