Masked Attention as a Mechanism for Improving Interpretability of Vision Transformers

arXiv:2404.18152v1 Announce Type: new
Abstract: Vision Transformers are at the heart of the current surge of interest in foundation models for histopathology. They process images by breaking them into smaller patches following a regular grid, regardless of their content. Yet, not all parts of an image are equally relevant for its understanding. This is particularly true in computational pathology where background is completely non-informative and may introduce artefacts that could mislead predictions. To address this issue, we propose a novel …

abstract arxiv attention breaking cs.cv current eess.iv foundation grid image images improving interpretability process them transformers type vision vision transformers