Toward Understanding the Disagreement Problem in Neural Network Feature Attribution

arXiv:2404.11330v1 Announce Type: new
Abstract: In recent years, neural networks have demonstrated their remarkable ability to discern intricate patterns and relationships from raw data. However, understanding the inner workings of these black box models remains challenging, yet crucial for high-stake decisions. Among the prominent approaches for explaining these black boxes are feature attribution methods, which assign relevance or contribution scores to each input variable for a model prediction. Despite the plethora of proposed techniques, ranging from gradient-based to backpropagation-based methods, …

abstract arxiv attribution black box black boxes box cs.lg data decisions feature however network networks neural network neural networks patterns raw relationships stat.ml type understanding