Evaluating Text-to-Visual Generation with Image-to-Text Generation

arXiv:2404.01291v1 Announce Type: cross
Abstract: Despite significant progress in generative AI, comprehensive evaluation remains challenging because of the lack of effective metrics and standardized benchmarks. For instance, the widely-used CLIPScore measures the alignment between a (generated) image and text prompt, but it fails to produce reliable scores for complex prompts involving compositions of objects, attributes, and relations. One reason is that text encoders of CLIP can notoriously act as a "bag of words", conflating prompts such as "the horse is …

abstract alignment arxiv benchmarks cs.ai cs.cl cs.cv cs.lg cs.mm evaluation generated generative image image-to-text instance metrics objects progress prompt prompts text text generation type visual