EncT5: A Framework for Fine-tuning T5 as Non-autoregressive Models. (arXiv:2110.08426v2 [cs.CL] UPDATED)

Pre-trained encoder-decoder transformer architectures have become
increasingly popular recently with the advent of T5 models. T5 has also become
more favorable over other architectures like BERT due to the amount of data
that it is pre-trained on, increased scale of model parameter sizes and easy
applicability to a diverse set of tasks due to the generative nature of the
model. While being able to generalize to a wide variety of tasks, it is not
clear that encoder-decoder architectures are the …

arxiv autoregressive models fine-tuning framework