Scalable and Robust Transformer Decoders for Interpretable Image Classification with Foundation Models

arXiv:2403.04125v1 Announce Type: new
Abstract: Interpretable computer vision models can produce transparent predictions, where the features of an image are compared with prototypes from a training dataset and the similarity between them forms a basis for classification. Nevertheless these methods are computationally expensive to train, introduce additional complexity and may require domain knowledge to adapt hyper-parameters to a new dataset. Inspired by developments in object detection, segmentation and large-scale self-supervised foundation vision models, we introduce Component Features (ComFe), a novel …

abstract arxiv classification complexity computer computer vision cs.cv dataset features forms foundation image predictions robust scalable them train training transformer type vision vision models