SHAP values for feature attribution

Description

Attributes a model's prediction to its input features using Shapley values from game theory, explaining both individual decisions and global model behaviour.

SHAP assigns each feature a contribution to a prediction, relative to a baseline, such that contributions sum to the prediction and satisfy fairness axioms from cooperative game theory. Aggregated across cases, SHAP values show global importance and the direction of effects; for single cases, they explain why a user received a particular recommendation or risk score. In a UX audit of an AI system, SHAP reveals whether models rely on features users would find inappropriate or opaque, and supplies raw material for user-facing explanations. Rigour requires understanding that SHAP explains the model, not reality; correlated features distort attributions; and the choice of background data affects results.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Obtain access to the model and representative data, and define the prediction to explain.
  2. Choose an explainer (TreeSHAP for tree models, KernelSHAP or DeepSHAP otherwise) and a background dataset of 100-1,000 representative cases.
  3. Compute SHAP values for a sample of predictions and plot global summaries (beeswarm, mean absolute SHAP).
  4. Review individual explanations for key user cases, especially edge cases and complaints.
  5. Flag features that are sensitive, proxies for protected attributes, or misaligned with user expectations.
  6. Translate findings into design and governance actions, and test candidate user-facing explanations with users.

Sources

  • The Alignment Problem (Brian Christian)
  • Artificial Intelligence for Humans (Jeff Heaton)
  • Methods Atlas (Vivek Gopinath)
  • Lundberg & Lee (2017) A unified approach to interpreting model predictions
  • Interpretable Machine Learning (Christoph Molnar)
  • The Alignment Problem (Christian)
  • AI for Humans (Heaton)
  • Methods Atlas (Gopinath)
  • The Inquiry Series, Ch 58 — AI Explainability & Value Alignment Audit

Also known as

  • SHapley Additive exPlanations
  • Shapley value attribution

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning