BOAT: Bilateral Local Attention Vision Transformer. (arXiv:2201.13027v2 [cs.CV] UPDATED)

Vision Transformers achieved outstanding performance in many computer vision
tasks. Early Vision Transformers such as ViT and DeiT adopt global
self-attention, which is computationally expensive when the number of patches
is large. To improve efficiency, recent Vision Transformers adopt local
self-attention mechanisms, where self-attention is computed within local
windows. Despite the fact that window-based local self-attention significantly
boosts efficiency, it fails to capture the relationships between distant but
similar patches in the image plane. To overcome this limitation of image-space …

arxiv attention local attention transformer vision