V-FLUTE: Visual Figurative Language Understanding with Textual Explanations

arXiv:2405.01474v1 Announce Type: new
Abstract: Large Vision-Language models (VLMs) have demonstrated strong reasoning capabilities in tasks requiring a fine-grained understanding of literal images and text, such as visual question-answering or visual entailment. However, there has been little exploration of these models' capabilities when presented with images and captions containing figurative phenomena such as metaphors or humor, the meaning of which is often implicit. To close this gap, we propose a new task and a high-quality dataset: Visual Figurative Language Understanding …

abstract arxiv capabilities captions cs.ai cs.cl cs.cv exploration fine-grained however images language language models language understanding question reasoning tasks text textual type understanding vision vision-language vision-language models visual vlms