Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots

arXiv:2404.18702v1 Announce Type: new
Abstract: The adoption of artificial intelligence (AI) across industries has led to the widespread use of complex black-box models and interpretation tools for decision making. This paper proposes an adversarial framework to uncover the vulnerability of permutation-based interpretation methods for machine learning tasks, with a particular focus on partial dependence (PD) plots. This adversarial framework modifies the original black box model to manipulate its predictions for instances in the extrapolation domain. As a result, it produces …

abstract adoption adversarial adversarial attacks artificial artificial intelligence arxiv attacks box cs.lg decision decision making framework industries intelligence interpretation machine machine learning making paper plots stat.ap stat.ml tools trust type vulnerability