Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations. (arXiv:2211.12486v1 [cs.LG])

While the evaluation of explanations is an important step towards trustworthy
models, it needs to be done carefully, and the employed metrics need to be
well-understood. Specifically model randomization testing is often
overestimated and regarded as a sole criterion for selecting or discarding
certain explanation methods. To address shortcomings of this test, we start by
observing an experimental gap in the ranking of explanation methods between
randomization-based sanity checks [1] and model output faithfulness measures
(e.g. [25]). We identify limitations …

arxiv checks deep neural network network neural network randomization top