all AI news
GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers
April 1, 2024, 4:43 a.m. | Takeru Miyato, Bernhard Jaeger, Max Welling, Andreas Geiger
cs.LG updates on arXiv.org arxiv.org
Abstract: As transformers are equivariant to the permutation of input tokens, encoding the positional information of tokens is necessary for many tasks. However, since existing positional encoding schemes have been initially designed for NLP tasks, their suitability for vision tasks, which typically exhibit different structural properties in their data, is questionable. We argue that existing positional encoding schemes are suboptimal for 3D vision tasks, as they do not respect their underlying 3D geometric structure. Based on …
abstract arxiv attention cs.ai cs.cv cs.lg encoding geometry gta however information nlp positional encoding stat.ml tasks tokens transformers type view vision
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
#13721 - Data Engineer - AI Model Testing
@ Qualitest | Miami, Florida, United States
Elasticsearch Administrator
@ ManTech | 201BF - Customer Site, Chantilly, VA