Improving Explicit Spatial Relationships in Text-to-Image Generation through an Automatically Derived Dataset

arXiv:2403.00587v1 Announce Type: new
Abstract: Existing work has observed that current text-to-image systems do not accurately reflect explicit spatial relations between objects such as 'left of' or 'below'. We hypothesize that this is because explicit spatial relations rarely appear in the image captions used to train these models. We propose an automatic method that, given existing images, generates synthetic captions that contain 14 explicit spatial relations. We introduce the Spatial Relation for Generation (SR4G) dataset, which contains 9.9 millions image-caption …

abstract arxiv captions cs.ai cs.cv current dataset image image generation objects relations relationships spatial systems text text-to-image through train type work