K-Nearest Neighbours (KNN)

Description

Classifies or predicts a case from the most similar past cases, a simple way to find "users like this one".

KNN stores the training data and, for a new case, finds the k most similar records using a distance metric on standardised variables, then predicts by majority vote or average. It makes no assumptions about the shape of relationships. Bruce and Bruce show it also works as a feature engine, adding a "neighbour score" to other models. For UX, the same similarity logic powers lookalike recruiting, matching a new user to behavioural archetypes, and finding comparable sessions to review.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Hours

Steps

  1. Choose and standardise predictor variables (z-scores) so none dominates distance.
  2. Encode categories with one-hot variables or use Gower distance for mixed data.
  3. Choose k by cross-validation.
  4. Classify or predict new cases from their neighbours.
  5. Inspect the nearest neighbours to explain individual predictions.

Sources

  • Practical Statistics for Data Scientists (Bruce & Bruce)
  • Myatt, G., Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining (2007)

Also known as

  • KNN
  • Nearest-neighbour classification

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning