Improving Multilingual Translation by Representation and Gradient Regularization. (arXiv:2109.04778v2 [cs.CL] UPDATED)

Multilingual Neural Machine Translation (NMT) enables one model to serve all
translation directions, including ones that are unseen during training, i.e.
zero-shot translation. Despite being theoretically attractive, current models
often produce low quality translations -- commonly failing to even produce
outputs in the right target language. In this work, we observe that off-target
translation is dominant even in strong multilingual systems, trained on massive
multilingual corpora. To address this issue, we propose a joint approach to
regularize NMT models at …

arxiv gradient translation