Nearest Neighbor Non-autoregressive Text Generation. (arXiv:2208.12496v1 [cs.CL])

Non-autoregressive (NAR) models can generate sentences with less computation
than autoregressive models but sacrifice generation quality. Previous studies
addressed this issue through iterative decoding. This study proposes using
nearest neighbors as the initial state of an NAR decoder and editing them
iteratively. We present a novel training strategy to learn the edit operations
on neighbors to improve NAR text generation. Experimental results show that the
proposed method (NeighborEdit) achieves higher translation quality (1.69 points
higher than the vanilla Transformer) with …

arxiv generation text text generation