Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation

arXiv:2405.01280v1 Announce Type: new
Abstract: Non-autoregressive (NAR) language models are known for their low latency in neural machine translation (NMT). However, a performance gap exists between NAR and autoregressive models due to the large decoding space and difficulty in capturing dependency between target words accurately. Compounding this, preparing appropriate training data for NAR models is a non-trivial task, often exacerbating exposure bias. To address these challenges, we apply reinforcement learning (RL) to Levenshtein Transformer, a representative edit-based NAR model, demonstrating …

abstract arxiv autoregressive autoregressive models cs.cl data decoding edit gap however language language models latency low low latency machine machine translation neural machine translation performance reinforcement reinforcement learning space training training data translation type words