Mixture of Regressions (Latent-Class Regression, Mixtures of Experts)

Description

Fit several regression or logistic models at once, one per hidden subgroup, with EM estimating each case's membership, to reveal segments that respond to drivers differently.

A single regression assumes everyone responds to predictors the same way. A mixture of K regressions gives each latent component its own coefficients, with shared or separate noise and mixing proportions, fitted by expectation-maximisation. In the E step each component's responsibility for each case is computed from the current fits; in the M step mixing proportions become average responsibilities and each component is refitted by weighted least squares (or weighted logistic regression) with responsibilities as case weights. When mixing proportions depend on inputs through a gating function the model becomes a mixture of experts, in which components specialise in regions of input space; nesting gates gives a soft, probabilistic decision tree. Such mixtures can represent multimodal outcomes a single regression cannot. For UX, the components are candidate segments defined by how satisfaction or conversion responds to drivers, not by who users are; this application is an adaptation rather than part of the source text.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Choose the component model (linear for a continuous outcome such as satisfaction, logistic for a binary one such as conversion) and a range of candidate K.
  2. Initialise component coefficients, mixing proportions and noise variance, using several random starts.
  3. E step: compute each component's responsibility for each case from the current fits.
  4. M step: set mixing proportions to mean responsibilities, refit each component by weighted least squares or weighted logistic regression, and update the noise variance.
  5. Iterate until the log-likelihood converges, keep the best of the starts, and choose K with an information criterion and interpretability.
  6. Optionally model membership from case attributes (mixture of experts) so new users can be assigned to segments.
  7. Inspect each component's coefficients and the cases it claims, and validate segments on held-out data.

Sources

  • Pattern Recognition and Machine Learning (Bishop, 2006)
  • Finite Mixture Models (McLachlan & Peel, 2000)
  • Applied Bayesian Modelling (Congdon, 2003)
  • Quantitative Models in Marketing Research (Franses & Paap, 2001)
  • Bayesian Statistical Modelling (Congdon, 2006)

Also known as

  • Mixture of linear and logistic regressions
  • Mixture of regressions
  • Latent class regression
  • Clusterwise regression
  • Finite mixture regression
  • Mixture of experts
  • Hierarchical mixture of experts

Categories

  • Analytics & Behaviour
  • Statistics & Modelling