all AI news
Unveiling Transformers with LEGO: a synthetic reasoning task. (arXiv:2206.04301v1 [cs.LG])
June 10, 2022, 1:10 a.m. | Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Tal Wagner
cs.LG updates on arXiv.org arxiv.org
We propose a synthetic task, LEGO (Learning Equality and Group Operations),
that encapsulates the problem of following a chain of reasoning, and we study
how the transformer architecture learns this task. We pay special attention to
data effects such as pretraining (on seemingly unrelated NLP tasks) and dataset
composition (e.g., differing chain length at training and test time), as well
as architectural variants such as weight-tied layers or adding convolutional
components. We study how the trained models eventually succeed at …
More from arxiv.org / cs.LG updates on arXiv.org
Regularization by Texts for Latent Diffusion Inverse Solvers
1 day, 3 hours ago |
arxiv.org
When can transformers reason with abstract symbols?
1 day, 3 hours ago |
arxiv.org
Jobs in AI, ML, Big Data
Data Scientist (m/f/x/d)
@ Symanto Research GmbH & Co. KG | Spain, Germany
Data Science Sustainability Co-Op (Summer & Fall 2024)
@ O-I | Perrysburg, OH, United States
Research Scientist
@ Chevron Phillips Chemical Company | USA: Kingwood, TX, US, 77339
Data Scientist Python (Django) (m/f/d)
@ RoomPriceGenie | Hybrid Mannheim, Remote DACH, Remote Germany
Operational Transformation & Strategy - Data Operations - Associate
@ JPMorgan Chase & Co. | Mumbai, Maharashtra, India
Senior Data Scientist
@ Rocket Travel | Chicago, IL USA