Analysis of Levenshtein Transformer's Decoder and Its Variants

arXiv:2402.12249v1 Announce Type: new
Abstract: Levenshtein transformer (LevT) is a non-autoregressive machine translation model with high decoding efficiency and comparable translation quality in terms of bleu score, due to its parallel decoding and iterative refinement procedure. Are there any deficiencies of its translations and what improvements could be made? In this report, we focus on LevT's decoder and analyse the decoding results length, subword generation, and deletion module's capability. We hope to identify weaknesses of the decoder for future improvements. …

abstract analysis arxiv bleu cs.cl decoder decoding efficiency improvements iterative machine machine translation quality report terms transformer translation type variants