SP-ViT: Learning 2D Spatial Priors for Vision Transformers. (arXiv:2206.07662v1 [cs.CV])

Web: http://arxiv.org/abs/2206.07662

Recently, transformers have shown great potential in image classification and
established state-of-the-art results on the ImageNet benchmark. However,
compared to CNNs, transformers converge slowly and are prone to overfitting in
low-data regimes due to the lack of spatial inductive biases. Such spatial
inductive biases can be especially beneficial since the 2D structure of an
input image is not well preserved in transformers. In this work, we present
Spatial Prior-enhanced Self-Attention (SP-SA), a novel variant of vanilla
Self-Attention (SA) tailored for …

arxiv cv learning transformers vision