Exploring Representational Disparities Between Multilingual and Bilingual Translation Models

arXiv:2305.14230v2 Announce Type: replace
Abstract: Multilingual machine translation has proven immensely useful for both parameter efficiency and overall performance across many language pairs via complete multilingual parameter sharing. However, some language pairs in multilingual models can see worse performance than in bilingual models, especially in the one-to-many translation setting. Motivated by their empirical differences, we examine the geometric differences in representations from bilingual models versus those from one-to-many multilingual models. Specifically, we compute the isotropy of these representations using intrinsic …

abstract arxiv bilingual cs.cl efficiency however language machine machine translation multilingual multilingual models performance translation type via