Getting it Right: Improving Spatial Consistency in Text-to-Image Models

arXiv:2404.01197v1 Announce Type: new
Abstract: One of the key shortcomings in current text-to-image (T2I) models is their inability to consistently generate images which faithfully follow the spatial relationships specified in the text prompt. In this paper, we offer a comprehensive investigation of this limitation, while also developing datasets and methods that achieve state-of-the-art performance. First, we find that current vision-language datasets do not represent spatial relationships well enough; to alleviate this bottleneck, we create SPRIGHT, the first spatially-focused, large scale …

abstract arxiv cs.cv current datasets generate image images improving investigation key paper prompt relationships spatial text text-to-image the key type