all AI news
Reasoning in Transformers - Mitigating Spurious Correlations and Reasoning Shortcuts
March 19, 2024, 4:41 a.m. | Daniel Enstr\"om, Viktor Kjellberg, Moa Johansson
cs.LG updates on arXiv.org arxiv.org
Abstract: Transformer language models are neural networks used for a wide variety of tasks concerning natural language, including some that also require logical reasoning. However, a transformer model may easily learn spurious patterns in the data, short-circuiting actual reasoning. In this paper we investigate to what extent transformers can be trained to a) approximate reasoning in propositional logic while b) avoiding known reasoning shortcuts via spurious correlations in the training data. To do so, we use …
abstract arxiv correlations cs.cl cs.lg data however language language models learn natural natural language networks neural networks paper patterns reasoning tasks transformer transformer language models transformer model transformers type
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Software Engineer for AI Training Data (School Specific)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Python)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Tier 2)
@ G2i Inc | Remote
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US