Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?

arXiv:2403.04792v1 Announce Type: cross
Abstract: Large language models hold significant promise in multilingual applications. However, inherent biases stemming from predominantly English-centric pre-training have led to the widespread practice of pre-translation, i.e., translating non-English inputs to English before inference, leading to complexity and information loss. This study re-evaluates the need for pre-translation in the context of PaLM2 models (Anil et al., 2023), which have been established as highly performant in multilingual tasks. We offer a comprehensive investigation across 108 languages and …

abstract applications arxiv biases breaking complexity cs.cl cs.lg english however inference information inputs language language models large language large language models llm llm applications loss multilingual multilingual llm practice pre-training stemming study training translation type