Compositional Text-to-Image Generation with Dense Blob Representations

arXiv:2405.08246v1 Announce Type: cross
Abstract: Existing text-to-image models struggle to follow complex text prompts, raising the need for extra grounding inputs for better controllability. In this work, we propose to decompose a scene into visual primitives - denoted as dense blob representations - that contain fine-grained details of the scene while being modular, human-interpretable, and easy-to-construct. Based on blob representations, we develop a blob-grounded text-to-image diffusion model, termed BlobGEN, for compositional generation. Particularly, we introduce a new masked cross-attention module …

arxiv cs.ai cs.cv cs.lg image image generation text text-to-image type