Learned Queries for Efficient Local Attention. (arXiv:2112.11435v2 [cs.CV] UPDATED)

Vision Transformers (ViT) serve as powerful vision models. Unlike
convolutional neural networks, which dominated vision research in previous
years, vision transformers enjoy the ability to capture long-range dependencies
in the data. Nonetheless, an integral part of any transformer architecture, the
self-attention mechanism, suffers from high latency and inefficient memory
utilization, making it less suitable for high-resolution input images. To
alleviate these shortcomings, hierarchical vision models locally employ
self-attention on non-interleaving windows. This relaxation reduces the
complexity to be linear in …

arxiv attention cv local attention