SambaLingo: Teaching Large Language Models New Languages

arXiv:2404.05829v1 Announce Type: cross
Abstract: Despite the widespread availability of LLMs, there remains a substantial gap in their capabilities and availability across diverse languages. One approach to address these issues has been to take an existing pre-trained LLM and continue to train it on new languages. While prior works have experimented with language adaptation, many questions around best practices and methodology have not been covered. In this paper, we present a comprehensive investigation into the adaptation of LLMs to new …

abstract arxiv availability capabilities cs.ai cs.cl cs.lg diverse gap language language models languages large language large language models llm llms prior teaching train type