Analyzing Local Representations of Self-supervised Vision Transformers

arXiv:2401.00463v2 Announce Type: replace
Abstract: In this paper, we present a comparative analysis of various self-supervised Vision Transformers (ViTs), focusing on their local representative power. Inspired by large language models, we examine the abilities of ViTs to perform various computer vision tasks with little to no fine-tuning. We design evaluation framework to analyze the quality of local, i.e.\ patch-level, representations in the context of few-shot semantic segmentation, instance identification, object retrieval and tracking. We discover that contrastive learning based methods …

abstract analysis analyze arxiv comparative analysis computer computer vision cs.cv design evaluation fine-tuning framework language language models large language large language models paper power tasks transformers type vision vision transformers