I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures

arXiv:2404.17918v1 Announce Type: new
Abstract: Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a …

abstract architectures arxiv attention bridge capabilities cs.cl inference machine machine translation modular paradigm small study training translation type