Quantifying Multilingual Performance of Large Language Models Across Languages

arXiv:2404.11553v1 Announce Type: new
Abstract: The training process of Large Language Models (LLMs) requires extensive text corpus. However, these data are often unevenly distributed in different languages. As a result, LLMs perform well on common languages, such as English, German, and French, but perform poorly on low-resource languages. However, currently there is no work to quantitatively measure the performance of LLMs in low-resource languages. To fill this gap, we proposed the Language Ranker that aims to benchmark and rank different …

abstract arxiv cs.ai cs.cl cs.lg data distributed english french german however language language models languages large language large language models llms low multilingual performance process text training type