Web: http://arxiv.org/abs/2205.02023

May 5, 2022, 1:11 a.m. | Karolina Stańczak, Edoardo Ponti, Lucas Torroba Hennigen, Ryan Cotterell, Isabelle Augenstein

cs.CL updates on arXiv.org arxiv.org

The success of multilingual pre-trained models is underpinned by their
ability to learn representations shared by multiple languages even in absence
of any explicit supervision. However, it remains unclear how these models learn
to generalise across languages. In this work, we conjecture that multilingual
pre-trained models can derive language-universal abstractions about grammar. In
particular, we investigate whether morphosyntactic information is encoded in
the same subset of neurons in different languages. We conduct the first
large-scale empirical study over 43 languages …

arxiv models neurons

Data Analyst, Patagonia Action Works

@ Patagonia | Remote

Data & Insights Strategy & Innovation General Manager

@ Chevron Services Company, a division of Chevron U.S.A Inc. | Houston, TX

Faculty members in Research areas such as Bayesian and Spatial Statistics; Data Privacy and Security; AI/ML; NLP; Image and Video Data Analysis

@ Ahmedabad University | Ahmedabad, India

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC