FasterViT: Fast Vision Transformers with Hierarchical Attention

arXiv:2306.06189v2 Announce Type: replace-cross
Abstract: We design a new family of hybrid CNN-ViT neural networks, named FasterViT, with a focus on high image throughput for computer vision (CV) applications. FasterViT combines the benefits of fast local representation learning in CNNs and global modeling properties in ViT. Our newly introduced Hierarchical Attention (HAT) approach decomposes global self-attention with quadratic complexity into a multi-level attention with reduced computational costs. We benefit from efficient window-based self-attention. Each window has access to dedicated carrier …

arxiv attention cs.ai cs.cv cs.lg hierarchical transformers type vision vision transformers