VLCap: Vision-Language with Contrastive Learning for Coherent Video Paragraph Captioning. (arXiv:2206.12972v2 [cs.CV] UPDATED)

In this paper, we leverage the human perceiving process, that involves vision
and language interaction, to generate a coherent paragraph description of
untrimmed videos. We propose vision-language (VL) features consisting of two
modalities, i.e., (i) vision modality to capture global visual content of the
entire scene and (ii) language modality to extract scene elements description
of both human and non-human objects (e.g. animals, vehicles, etc), visual and
non-visual elements (e.g. relations, activities, etc). Furthermore, we propose
to train our proposed …

arxiv captioning cv language learning video vision