An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM

arXiv:2403.18406v1 Announce Type: cross
Abstract: Stimulated by the sophisticated reasoning capabilities of recent Large Language Models (LLMs), a variety of strategies for bridging video modality have been devised. A prominent strategy involves Video Language Models (VideoLMs), which train a learnable interface with video data to connect advanced vision encoders with LLMs. Recently, an alternative strategy has surfaced, employing readily available foundation models, such as VideoLMs and LLMs, across multiple stages for modality bridging. In this study, we introduce a simple …

abstract arxiv capabilities cs.ai cs.cl cs.cv cs.lg data grid image language language models large language large language models llms question question answering reasoning strategies strategy train type video video data vlm zero-shot