TESS: Text-to-Text Self-Conditioned Simplex Diffusion

arXiv:2305.08379v2 Announce Type: replace-cross
Abstract: Diffusion models have emerged as a powerful paradigm for generation, obtaining strong performance in various continuous domains. However, applying continuous diffusion models to natural language remains challenging due to its discrete nature and the need for a large number of diffusion steps to generate text, making diffusion-based generation expensive. In this work, we propose Text-to-text Self-conditioned Simplex Diffusion (TESS), a text diffusion model that is fully non-autoregressive, employs a new form of self-conditioning, and applies …

arxiv cs.cl cs.lg diffusion tess text text-to-text type