Computer Vision Meetup: Improved Visual Grounding through Self-Consistent Explanations

Vision-and-language models that are trained to associate images with text have shown to be effective for many tasks, including object detection and image segmentation. In this talk, we will discuss how to enhance vision-and-language models’ ability to localize objects in images by fine-tuning them for self-consistent visual explanations. We propose a method that augments text-image datasets with paraphrases using a large language model and employs SelfEQ, a weakly-supervised strategy that promotes self-consistency in visual explanation maps. This approach broadens the …

ai computer computer vision computervision consistent datascience detection discuss fine-tuning image images language language models machinelearning meetup object objects segmentation talk tasks text them through tuning vision vision-and-language visual will