Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?

arXiv:2406.07546v1 Announce Type: cross
Abstract: We present a novel task and benchmark for evaluating the ability of text-to-image(T2I) generation models to produce images that fit commonsense in real life, which we call Commonsense-T2I. Given two adversarial text prompts containing an identical set of action words with minor differences, such as "a lightbulb without electricity" v.s. "a lightbulb with electricity", we evaluate whether T2I models can conduct visual-commonsense reasoning, e.g. produce images that fit "the lightbulb is unlit" vs. "the lightbulb …

abstract action adversarial arxiv benchmark call challenge commonsense cs.ai cs.cl cs.cv differences image image generation image generation models images life novel prompts set text text-to-image type words