Improving Interpretation Faithfulness for Vision Transformers

arXiv:2311.17983v2 Announce Type: replace-cross
Abstract: Vision Transformers (ViTs) have achieved state-of-the-art performance for various vision tasks. One reason behind the success lies in their ability to provide plausible innate explanations for the behavior of neural architectures. However, ViTs suffer from issues with explanation faithfulness, as their focal points are fragile to adversarial attacks and can be easily changed with even slight perturbations on the input image. In this paper, we propose a rigorous approach to mitigate these issues by introducing …

abstract adversarial adversarial attacks architectures art arxiv attacks behavior cs.ai cs.cv cs.lg however improving interpretation lies neural architectures performance reason state success tasks transformers type vision vision transformers