Understanding and Mitigating Compositional Issues in Text-to-Image Generative Models

arXiv:2406.07844v1 Announce Type: new
Abstract: Recent text-to-image diffusion-based generative models have the stunning ability to generate highly detailed and photo-realistic images and achieve state-of-the-art low FID scores on challenging image generation benchmarks. However, one of the primary failure modes of these text-to-image generative models is in composing attributes, objects, and their associated relationships accurately into an image. In our paper, we investigate this compositionality-based failure mode and highlight that imperfect text conditioning with CLIP text-encoder is one of the primary …

abstract art arxiv attributes benchmarks cs.cv diffusion failure generate generative generative models however image image diffusion image generation images low objects photo state text text-to-image type understanding