Description
Attributes a model's prediction to its input features using Shapley values from game theory, explaining both individual decisions and global model behaviour.
SHAP assigns each feature a contribution to a prediction, relative to a baseline, such that contributions sum to the prediction and satisfy fairness axioms from cooperative game theory. Aggregated across cases, SHAP values show global importance and the direction of effects; for single cases, they explain why a user received a particular recommendation or risk score. In a UX audit of an AI system, SHAP reveals whether models rely on features users would find inappropriate or opaque, and supplies raw material for user-facing explanations. Rigour requires understanding that SHAP explains the model, not reality; correlated features distort attributions; and the choice of background data affects results.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Analytical / expert
- Effort
- Days
Steps
- Obtain access to the model and representative data, and define the prediction to explain.
- Choose an explainer (TreeSHAP for tree models, KernelSHAP or DeepSHAP otherwise) and a background dataset of 100-1,000 representative cases.
- Compute SHAP values for a sample of predictions and plot global summaries (beeswarm, mean absolute SHAP).
- Review individual explanations for key user cases, especially edge cases and complaints.
- Flag features that are sensitive, proxies for protected attributes, or misaligned with user expectations.
- Translate findings into design and governance actions, and test candidate user-facing explanations with users.
Sources
- The Alignment Problem (Brian Christian)
- Artificial Intelligence for Humans (Jeff Heaton)
- Methods Atlas (Vivek Gopinath)
- Lundberg & Lee (2017) A unified approach to interpreting model predictions
- Interpretable Machine Learning (Christoph Molnar)
- The Alignment Problem (Christian)
- AI for Humans (Heaton)
- Methods Atlas (Gopinath)
- The Inquiry Series, Ch 58 — AI Explainability & Value Alignment Audit
Also known as
- SHapley Additive exPlanations
- Shapley value attribution
Categories
- Analytics & Behaviour
- Computational & Machine Learning