InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding

arXiv:2403.15377v1 Announce Type: new
Abstract: We introduce InternVideo2, a new video foundation model (ViFM) that achieves the state-of-the-art performance in action recognition, video-text tasks, and video-centric dialogue. Our approach employs a progressive training paradigm that unifies the different self- or weakly-supervised learning frameworks of masked video token reconstruction, cross-modal contrastive learning, and next token prediction. Different training stages would guide our model to capture different levels of structure and semantic information through different pretext tasks. At the data level, we …

arxiv cs.cv foundation multimodal scaling type understanding video video understanding