Can I Trust Your Answer? Visually Grounded Video Question Answering

arXiv:2309.01327v2 Announce Type: replace
Abstract: We study visually grounded VideoQA in response to the emerging trends of utilizing pretraining techniques for video-language understanding. Specifically, by forcing vision-language models (VLMs) to answer questions and simultaneously provide visual evidence, we seek to ascertain the extent to which the predictions of such techniques are genuinely anchored in relevant video content, versus spurious correlations from language or irrelevant visual context. Towards this, we construct NExT-GQA -- an extension of NExT-QA with 10.5$K$ temporal grounding …

arxiv cs.ai cs.cv cs.mm question question answering trust type video