Investigating the Impact of Model Instability on Explanations and Uncertainty

arXiv:2402.13006v1 Announce Type: new
Abstract: Explainable AI methods facilitate the understanding of model behaviour, yet, small, imperceptible perturbations to inputs can vastly distort explanations. As these explanations are typically evaluated holistically, before model deployment, it is difficult to assess when a particular explanation is trustworthy. Some studies have tried to create confidence estimators for explanations, but none have investigated an existing link between uncertainty and explanation quality. We artificially simulate epistemic uncertainty in text input by introducing noise at inference …

abstract arxiv cs.cl cs.lg deployment explainable ai impact inputs model deployment small studies trustworthy type uncertainty understanding