Description
A high-accuracy model that builds many small trees in sequence, each correcting the last, for predicting churn, satisfaction or conversion.
Boosting fits a series of shallow trees where each new tree focuses on the errors of the ensemble so far. XGBoost adds regularisation and efficient computation, and it wins many prediction competitions on tabular data. Bruce and Bruce explain the key hyperparameters (learning rate, depth, subsampling, regularisation) and how to tune them with cross-validation to avoid overfitting. Paired with SHAP values, boosting becomes an explainable driver model for large behavioural datasets.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Unmigrated
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Weeks
Steps
- Prepare a tabular dataset of user features and an outcome.
- Split into training and test sets (or use cross-validation).
- Tune learning rate, tree depth, subsampling and regularisation.
- Evaluate on held-out data with AUC, precision and recall.
- Explain predictions with SHAP values and validate surprising drivers qualitatively.
Sources
- Practical Statistics for Data Scientists (Bruce & Bruce)
Also known as
- XGBoost
- Boosted trees
- GBM
Categories
- Analytics & Behaviour
- Computational & Machine Learning