Adversarial Testing for Visual Grounding via Image-Aware Property Reduction

arXiv:2403.01118v1 Announce Type: new
Abstract: Due to the advantages of fusing information from various modalities, multimodal learning is gaining increasing attention. Being a fundamental task of multimodal learning, Visual Grounding (VG), aims to locate objects in images through natural language expressions. Ensuring the quality of VG models presents significant challenges due to the complex nature of the task. In the black box scenario, existing adversarial testing techniques often fail to fully exploit the potential of both modalities of information. They …

abstract advantages adversarial arxiv attention challenges cs.ai cs.cv image images information language multimodal multimodal learning natural natural language objects property quality testing through type via visual