LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?

arXiv:2404.10763v1 Announce Type: cross
Abstract: Diffusion models have exhibited remarkable capabilities in text-to-image generation. However, their performance in image-to-text generation, specifically image captioning, has lagged behind Auto-Regressive (AR) models, casting doubt on their applicability for such tasks. In this work, we revisit diffusion models, highlighting their capacity for holistic context modeling and parallel decoding. With these benefits, diffusion models can alleviate the inherent limitations of AR methods, including their slow inference speed, error propagation, and unidirectional constraints. Furthermore, we identify …

abstract arxiv auto autoregressive capabilities capacity captioning cs.ai cs.cl cs.cv diffusion diffusion models highlighting however image image generation image-to-text performance tasks text text generation text-to-image type work