How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning

arXiv:2305.13286v2 Announce Type: replace
Abstract: Multilingual large language models (MLLMs) are jointly trained on data from many different languages such that representation of individual languages can benefit from other languages' data. Impressive performance on zero-shot cross-lingual transfer shows that these models are capable of exploiting data from other languages. Yet, it remains unclear to what extent, and under which conditions, languages rely on each other's data. In this study, we use TracIn (Pruthi et al., 2020), a training data attribution …

abstract arxiv benefit cross-lingual cs.cl data data sharing fine-tuning influence language language models languages large language large language models mllms multilingual performance replace representation shows studying transfer type zero-shot