[D] ICLR Outstanding Paper Awards. Congratulations!

[**Vision Transformers Need Registers**](https://openreview.net/forum?id=2dnO3LLiJ1)
*Timothée Darcet, Maxime Oquab, Julien Mairal, Piotr Bojanowski*

**Abstract:** Transformers have recently emerged as a powerful tool for learning visual representations. In this paper, we identify and characterize artifacts in feature maps of both supervised and self-supervised ViT networks. The artifacts correspond to high-norm tokens appearing during inference primarily in low-informative background areas of images, that are repurposed for internal computations. We propose a simple yet effective solution based on providing additional tokens to the input …

abstract feature identify images inference low machinelearning maps networks norm paper tokens tool transformers visual vit