Discriminative Probing and Tuning for Text-to-Image Generation

arXiv:2403.04321v1 Announce Type: new
Abstract: Despite advancements in text-to-image generation (T2I), prior methods often face text-image misalignment problems such as relation confusion in generated images. Existing solutions involve cross-attention manipulation for better compositional understanding or integrating large language models for improved layout planning. However, the inherent alignment capabilities of T2I models are still inadequate. By reviewing the link between generative and discriminative modeling, we posit that T2I models' discriminative abilities may reflect their text-image alignment proficiency during generation. In this …

abstract alignment arxiv attention capabilities cs.ai cs.cl cs.cv cs.mm face generated however image image generation images language language models large language large language models manipulation planning prior solutions text text-image text-to-image type understanding