UMAP visualisation

Description

Projecting high-dimensional behavioural or text-embedding features into two dimensions to see clusters and gradients among users, sessions or responses.

UMAP builds a weighted nearest-neighbour graph in the original feature space and finds a low-dimensional layout preserving its structure. It is faster than t-SNE and preserves more global structure, making it useful for visually inspecting behavioural segments or semantic clusters of open-text responses. Its key parameters are n_neighbors (local versus global emphasis, typically 15-50) and min_dist (how tightly points pack). Rigour comes from treating UMAP as exploratory: distances and cluster sizes in the plot are not directly interpretable, apparent clusters can be artefacts of parameters, and results vary with the random seed. Clusters should be confirmed with a model fitted in the original space and checked across several parameter settings.

Attributes

Loss type
Siloed
Stage
Discovery
Effort
Hours

Steps

  1. Prepare a standardised feature matrix or sentence embeddings and remove near-duplicate rows.
  2. Run UMAP with default parameters (n_neighbors 15, min_dist 0.1) and a fixed random seed.
  3. Repeat with n_neighbors of 5, 30 and 50 and at least three seeds, keeping only structure that persists.
  4. Colour points by known variables (plan type, tenure, GMM segment, sentiment) to interpret regions.
  5. Confirm apparent clusters with a model fitted in the original space (GMM, HDBSCAN) and report its fit statistics, not the picture.
  6. Annotate plots with caveats that axes and distances are not meaningful.

Sources

  • lifelines (survival)
  • scikit-learn (GMM, LDA, UMAP)
  • pandas (log parsing)
  • matplotlib
  • plotly
  • McInnes, L., Healy, J. & Melville, J. (2018) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv:1802.03426
  • The Inquiry Series, C4 — Behavioural Log Analysis — From Clickstreams to Personas

Also known as

  • Uniform Manifold Approximation and Projection
  • Non-linear dimensionality reduction

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning