Lossless Speedup of Autoregressive Translation with Generalized Aggressive Decoding. (arXiv:2203.16487v4 [cs.CL] UPDATED)

Different from previous work accelerating translation at the cost of quality
loss, we propose Generalized Aggressive Decoding (GAD) -- a novel decoding
paradigm for lossless speedup of autoregressive translation, through the
collaboration of autoregressive and non-autoregressive translation (NAT) of the
Transformer. At each decoding iteration, GAD aggressively decodes a number of
tokens with NAT as a draft and then verifies them in the autoregressive manner,
where only the tokens that pass the verification are accepted as decoded
tokens. GAD can …

arxiv translation