Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation. (arXiv:2208.09970v2 [stat.ME] UPDATED)

SHAP is a popular method for measuring variable importance in machine
learning models. In this paper, we study the algorithm used to estimate SHAP
scores and outline its connection to the functional ANOVA decomposition. We use
this connection to show that challenges in SHAP approximations largely relate
to the choice of a feature distribution and the number of $2^p$ ANOVA terms
estimated. We argue that the connection between machine learning explainability
and sensitivity analysis is illuminating in this case, but …

anova arxiv interpretation shap statistical