Description
Uses random forests to estimate which experience attributes most strongly predict an outcome like satisfaction or churn, with careful importance measures.
Key driver analysis asks which attributes (speed, support, reliability, ease) matter most for an overall outcome. Multiple regression struggles with correlated attributes and nonlinearity. Random forests capture interactions and nonlinear effects, and provide variable importance. However, default impurity-based importance is biased toward variables with many categories, and permutation importance is biased under correlation; conditional permutation importance or SHAP values with care, plus partial dependence plots, are needed. Drivers are predictive, not causal, and should be presented as such. Many dashboards show 'drivers' from naive correlations or default importances, the vestigial form.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Vestigial
- Goal
- Prioritise
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Collect outcome measures and candidate attributes (survey ratings, behavioural metrics) for a large sample (e.g. 500+).
- Fit a random forest with tuned parameters and evaluate out-of-sample predictive accuracy.
- Compute conditional permutation importance or SHAP-based importance, and compare with a relative weights or dominance analysis.
- Examine partial dependence and accumulated local effects plots to see shape (thresholds, saturation).
- Check stability with bootstrapped importance rankings.
- Present drivers as predictive associations, then test top drivers causally where possible.
Sources
- Breiman, Random forests (2001), Machine Learning
- Strobl et al., Conditional variable importance for random forests (2008), BMC Bioinformatics
- Molnar, Interpretable Machine Learning (2nd ed., 2022)
- Practical Statistics for Data Scientists (Bruce & Bruce)
- Factor and Discriminant Analysis (Sauro & Lewis, MeasuringU training)
Also known as
- Random forests for UX drivers
- Driver analysis
- Relative importance analysis
Categories
- Analytics & Behaviour
- Computational & Machine Learning