VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives. (arXiv:2206.11212v1 [cs.CV])

Web: http://arxiv.org/abs/2206.11212

Many past works aim to improve visual reasoning in models by supervising
feature importance (estimated by model explanation techniques) with human
annotations such as highlights of important image regions. However, recent work
has shown that performance gains from feature importance (FI) supervision for
Visual Question Answering (VQA) tasks persist even with random supervision,
suggesting that these methods do not meaningfully align model FI with human FI.
In this paper, we show that model FI supervision can meaningfully improve VQA
model …

arxiv cv feature