FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

arXiv:2404.14715v1 Announce Type: new
Abstract: Recent progress in large-scale pre-training has led to the development of advanced vision-language models (VLMs) with remarkable proficiency in comprehending and generating multimodal content. Despite the impressive ability to perform complex reasoning for VLMs, current models often struggle to effectively and precisely capture the compositional information on both the image and text sides. To address this, we propose FineMatch, a new aspect-based fine-grained text and image matching benchmark, focusing on text and image mismatch detection …

abstract advanced advanced vision arxiv cs.cl cs.cv current detection development fine-grained image language language models multimodal multimodal content pre-training progress reasoning scale struggle text training type vision vision-language vision-language models vlms