Independent component analysis

Description

Separates several independent sources of variation mixed in correlated measurements by finding the rotation that makes estimated sources statistically independent, not merely uncorrelated.

Multivariate measurements are modelled as a linear mixture of a few underlying sources plus noise. Principal components analysis can count the sources from dominant eigenvalues, but its axes are distorted whenever the source patterns are not orthogonal, so each component may blend causes. Varimax-type rotation assumes each source touches a separate subset of variables. Blind source separation instead whitens the data and searches for the orthogonal rotation that minimises fourth-order cross-cumulants (the JADE criterion). This identifies the sources uniquely provided at most one is Gaussian. The result is a mixing matrix showing how each source shows up across variables and a score for each source per case. ICA can be read as another form of factor rotation; which rotation is more accurate depends on which assumptions hold, so the non-Gaussianity assumption should be checked.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Run PCA and count dominant eigenvalues to set the number of sources p; estimate noise variance from the remaining small eigenvalues.
  2. Whiten the data using the first p principal components.
  3. Estimate the rotation that minimises fourth-order cross-cumulants (JADE) or use another ICA algorithm such as FastICA.
  4. Compute the mixing matrix and recover each source's score per case by regression.
  5. Plot histograms of the estimated sources and confirm that no more than one looks Gaussian.
  6. Display each source pattern on a diagram of the product or journey so domain experts can name its cause, and compare with a varimax solution.

Sources

  • Handbook of Data Mining (Ye, 2003)
  • Information Theory, Inference, and Learning Algorithms (MacKay, 2003)
  • Signal Processing for Neuroscientists (van Drongelen, 2018)

Also known as

  • ICA
  • Blind source separation of variation patterns
  • Blind separation
  • JADE

Categories

  • Analytics & Behaviour
  • Statistics & Modelling