Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation. (arXiv:2209.02821v1 [cs.CL])

We propose a two-stage training approach for developing a single NMT model to
translate unseen languages both to and from English. For the first stage, we
initialize an encoder-decoder model to pretrained XLM-R and RoBERTa weights,
then perform multilingual fine-tuning on parallel data in 25 languages to
English. We find this model can generalize to zero-shot translations on unseen
languages. For the second stage, we leverage this generalization ability to
generate synthetic parallel data from monolingual datasets, then train with …

arxiv translation unsupervised