Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following

arXiv:2311.17002v3 Announce Type: replace
Abstract: Existing text-to-image (T2I) diffusion models usually struggle in interpreting complex prompts, especially those with quantity, object-attribute binding, and multi-subject descriptions. In this work, we introduce a semantic panel as the middleware in decoding texts to images, supporting the generator to better follow instructions. The panel is obtained through arranging the visual concepts parsed from the input text by the aid of large language models, and then injected into the denoising network as a detailed control …

arxiv cs.cv diffusion image image diffusion text text-to-image type