Tower: An Open Multilingual Large Language Model for Translation-Related Tasks

arXiv:2402.17733v1 Announce Type: new
Abstract: While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and parallel data, creating TowerBase, followed by finetuning on instructions relevant for translation processes, creating TowerInstruct. Our final …

abstract arxiv cs.cl domain general language language model language models large language large language model large language models llms multilingual multiple paper recipe tasks translation type