Lossless Acceleration for Seq2seq Generation with Aggressive Decoding. (arXiv:2205.10350v1 [cs.CL])

We study lossless acceleration for seq2seq generation with a novel decoding
algorithm -- Aggressive Decoding. Unlike the previous efforts (e.g.,
non-autoregressive decoding) speeding up seq2seq generation at the cost of
quality loss, our approach aims to yield the identical (or better) generation
compared with autoregressive decoding but in a significant speedup, achieved by
innovative cooperation of aggressive decoding and verification that are both
efficient due to parallel computing.

We propose two Aggressive Decoding paradigms for 2 kinds of seq2seq tasks: …

arxiv generation seq2seq