Description
Score each session's combined telemetry for abnormality with Isolation Forest and Local Outlier Factor, without labels, and rank the most abnormal sessions into a human review queue.
Average metrics hide rare but severe sessions: broken flows, crashes, bot-like behaviour, extreme friction. Unsupervised outlier detection looks at many telemetry features together and asks which sessions do not resemble the rest. Isolation Forest builds random trees and scores a session as abnormal when it can be isolated in few splits; Local Outlier Factor compares a session's local density with that of its nearest neighbours. Neither needs labelled incidents. In the Rafiei and Jozranjbar workshop notebook, about fourteen standardised per-session features (duration, rage clicks, form retries, errors, latency, backtracking, crash indicator and others) feed both detectors with an expected anomaly share of 5 per cent, and the highest-scoring sessions become a triage list for replay and bug review. Hidden labels are used only to show precision and recall. The rigour lies in feature preparation, checking agreement between detectors, and treating flags as leads for human investigation rather than verdicts.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Siloed
- Stage
- Post-launch, Evaluation
- Data
- Quantitative
- Effort
- Days
Steps
- Build one row per session with behavioural and technical features (for example duration, rage clicks, form retries, error messages, latency, backtracking, help visits, crash indicator, task completion), drop duplicates and impute missing values with the median.
- Standardise all features so that no single scale, such as latency in milliseconds, dominates the distance and split calculations.
- Fit Isolation Forest and Local Outlier Factor (the notebook uses 35 neighbours) with an expected contamination of about 5 per cent, and keep both the binary flags and the continuous abnormality scores.
- If any labelled incidents or known bad sessions exist, compute precision, recall and confusion matrices, and tune for recall first when missed failures are costlier than wasted reviews.
- Check where the two detectors agree, and project sessions onto two principal components to see whether flagged sessions sit apart from the dense core.
- Sort sessions by anomaly score and review the top of the list (the notebook uses 12) with session replay, logs and support tickets, recording which are real defects, bots or merely unusual users, and feed that back into triage rules.
Sources
- AI-for-quantitative-UX-research (Rafiei & Jozranjbar, GitHub repository, 2026)
Also known as
- UX session anomaly detection
- Isolation Forest session triage
- Local Outlier Factor session screening
- Unsupervised telemetry anomaly triage
Categories
- Analytics & Behaviour
- Computational & Machine Learning