Description
Train a predictive model on task-level features and decompose its predictions with SHAP values to rank which UX factors most drive success or drop-off.
SHAP attributes each prediction to features using Shapley values from cooperative game theory, giving additive contributions that sum to the prediction. Vivek Gopinath's playbook trains a gradient boosting model on a 70/30 split using task time per step, error counts (slips, lapses, mistakes), SNR, TLX scores and retries, evaluates it on holdout data, checks calibration with reliability plots, and ranks features by mean absolute SHAP value. In the example, mistake rate (0.33) and SNR (0.27) dominate, followed by OTP time (0.18), frustration (0.13) and retries (0.09), guiding fix priority. Pitfalls are leakage from post-outcome features, overfitting (use holdout or cross-validation) and treating importance as causal; SHAP must be paired with experiments and with qualitative quotes explaining each top driver. In the Behavioral Spectral Framework, SHAP or Integrated Gradients explain which frequencies contributed to change.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Latent
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Assemble task-level features measured before the outcome (task time, error counts, SNR, TLX, retries).
- Train a gradient boosting model on a 70/30 split and evaluate on the holdout set.
- Check calibration with reliability plots and guard against leakage.
- Compute SHAP values and rank features by mean absolute SHAP.
- Attach qualitative quotes explaining the mechanism of each top driver.
- Prioritise fixes by driver rank and validate with experiments.
Sources
- Vivek Gopinath (2025), Forward-Thinking UX Research Playbook: A Mixed-Methods, Decision-Ready Approach from Prediction to Validation
- Vivek Gopinath (2025), From Stars to Signals: Translating Matteo Paz's Fourier-Wavelet Model into Behavioral and UX Research Methodology
Also known as
- SHapley Additive exPlanations
- SHAP feature importance
Categories
- Analytics & Behaviour
- Computational & Machine Learning