Self-generated Replay Memories for Continual Neural Machine Translation

arXiv:2403.13130v1 Announce Type: cross
Abstract: Modern Neural Machine Translation systems exhibit strong performance in several different languages and are constantly improving. Their ability to learn continuously is, however, still severely limited by the catastrophic forgetting issue. In this work, we leverage a key property of encoder-decoder Transformers, i.e. their generative ability, to propose a novel approach to continually learning Neural Machine Translation systems. We show how this can effectively learn on a stream of experiences comprising different languages, by leveraging …

arxiv continual cs.ai cs.cl cs.lg generated machine machine translation memories neural machine translation translation type