all AI news
Transformer Vs. MLP-Mixer Exponential Expressive Gap For NLP Problems. (arXiv:2208.08191v1 [cs.CL])
Aug. 18, 2022, 1:10 a.m. | Dan Navon, Alex M. Bronstein
cs.LG updates on arXiv.org arxiv.org
Vision-Transformers are widely used in various vision tasks. Meanwhile, there
is another line of works starting with the MLP-mixer trying to achieve similar
performance using mlp-based architectures. Interestingly, until now none
reported using them for NLP tasks, additionally until now non of those
mlp-based architectures claimed to achieve state-of-the-art in vision tasks. In
this paper, we analyze the expressive power of mlp-based architectures in
modeling dependencies between multiple different inputs simultaneously, and
show an exponential gap between the attention and …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Data Architect
@ Western Digital | San Jose, CA, United States
Senior Data Scientist GenAI (m/w/d)
@ Deutsche Telekom | Bonn, Deutschland
Senior Data Engineer, Telco (Remote)
@ Lightci | Toronto, Ontario
Consultant Data Architect/Engineer H/F - Innovative Tech
@ Devoteam | Lyon, France
(Senior) ML Engineer / Software Engineer Machine Learning & AI (m/f/x) onsite or remote (in Germany or Austria)
@ Scalable GmbH | Wien, Germany