Object-Attribute Binding in Text-to-Image Generation: Evaluation and Control

arXiv:2404.13766v1 Announce Type: new
Abstract: Current diffusion models create photorealistic images given a text prompt as input but struggle to correctly bind attributes mentioned in the text to the right objects in the image. This is evidenced by our novel image-graph alignment model called EPViT (Edge Prediction Vision Transformer) for the evaluation of image-text alignment. To alleviate the above problem, we propose focused cross-attention (FCA) that controls the visual attention maps by syntactic constraints found in the input sentence. Additionally, …

abstract alignment arxiv control create cs.cv current diffusion diffusion models edge evaluation graph image image generation images novel object objects photorealistic photorealistic images prediction prompt struggle text text-to-image transformer type vision