all AI news
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention During Vision Transformer Inference. (arXiv:2211.10526v1 [cs.CV])
Nov. 22, 2022, 2:12 a.m. | Haoran You, Yunyang Xiong, Xiaoliang Dai, Bichen Wu, Peizhao Zhang, Haoqi Fan, Peter Vajda, Yingyan Lin
cs.CV updates on arXiv.org arxiv.org
Vision Transformers (ViTs) have shown impressive performance but still
require a high computation cost as compared to convolutional neural networks
(CNNs), due to the global similarity measurements and thus a quadratic
complexity with the input tokens. Existing efficient ViTs adopt local attention
(e.g., Swin) or linear attention (e.g., Performer), which sacrifice ViTs'
capabilities of capturing either global or local context. In this work, we ask
an important research question: Can ViTs learn both global and local context
while being more …
angular arxiv attention inference linear self-attention transformer vision
More from arxiv.org / cs.CV updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Data Architect
@ Western Digital | San Jose, CA, United States
Senior Data Scientist GenAI (m/w/d)
@ Deutsche Telekom | Bonn, Deutschland
Senior Data Engineer, Telco (Remote)
@ Lightci | Toronto, Ontario
Consultant Data Architect/Engineer H/F - Innovative Tech
@ Devoteam | Lyon, France
(Senior) ML Engineer / Software Engineer Machine Learning & AI (m/f/x) onsite or remote (in Germany or Austria)
@ Scalable GmbH | Wien, Germany