An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models

arXiv:2403.20088v1 Announce Type: new
Abstract: The capacity and effectiveness of pre-trained multilingual models (MLMs) for zero-shot cross-lingual transfer is well established. However, phenomena of positive or negative transfer, and the effect of language choice still need to be fully understood, especially in the complex setting of massively multilingual LMs. We propose an \textit{efficient} method to study transfer language influence in zero-shot performance on another target language. Unlike previous work, our approach disentangles downstream tasks from language, using dedicated adapter units. …

arxiv cross-lingual cs.cl language language models multilingual studying transfer type