Understanding the role of FFNs in driving multilingual behaviour in LLMs

arXiv:2404.13855v1 Announce Type: new
Abstract: Multilingualism in Large Language Models (LLMs) is an yet under-explored area. In this paper, we conduct an in-depth analysis of the multilingual capabilities of a family of a Large Language Model, examining its architecture, activation patterns, and processing mechanisms across languages. We introduce novel metrics to probe the model's multilingual behaviour at different layers and shed light on the impact of architectural choices on multilingual processing.
Our findings reveal different patterns of multilinugal processing in …

abstract analysis architecture arxiv capabilities cs.cl driving family language language model language models languages large language large language model large language models llms multilingual multilingualism novel paper patterns processing role type understanding