VidLA: Video-Language Alignment at Scale

arXiv:2403.14870v1 Announce Type: cross
Abstract: In this paper, we propose VidLA, an approach for video-language alignment at scale. There are two major limitations of previous video-language alignment approaches. First, they do not capture both short-range and long-range temporal dependencies and typically employ complex hierarchical deep network architectures that are hard to integrate with existing pretrained image-text foundation models. To effectively address this limitation, we instead keep the network architecture simple and use a set of data tokens that operate at …

abstract alignment architectures arxiv cs.cl cs.cv cs.lg dependencies hierarchical language limitations major network paper scale temporal type video