VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing

arXiv:2403.02581v1 Announce Type: new
Abstract: Visual entailment (VE) is a multimodal reasoning task consisting of image-sentence pairs whereby a promise is defined by an image, and a hypothesis is described by a sentence. The goal is to predict whether the image semantically entails the sentence. VE systems have been widely adopted in many downstream tasks. Metamorphic testing is the commonest technique for AI algorithms, but it poses a significant challenge for VE testing. They either only consider perturbations on single …

abstract arxiv cs.cv cs.se hypothesis image multimodal object reasoning systems testing type via visual