Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners. (arXiv:2205.10747v2 [cs.CV] UPDATED)

The goal of this work is to build flexible video-language models that can
generalize to various video-to-text tasks from few examples, such as
domain-specific captioning, question answering, and future event prediction.
Existing few-shot video-language learners focus exclusively on the encoder,
resulting in the absence of a video-to-text decoder to handle generative tasks.
Video captioners have been pretrained on large-scale video-language datasets,
but they rely heavily on finetuning and lack the ability to generate text for
unseen tasks in a few-shot …

arxiv cv image language language models video