Multi-granularity Correspondence Learning from Long-term Noisy Videos. (arXiv:2401.16702v1 [cs.CV])

Existing video-language studies mainly focus on learning short video clips,
leaving long-term temporal dependencies rarely explored due to over-high
computational cost of modeling long videos. To address this issue, one feasible
solution is learning the correspondence between video clips and captions, which
however inevitably encounters the multi-granularity noisy correspondence (MNC)
problem. To be specific, MNC refers to the clip-caption misalignment
(coarse-grained) and frame-word misalignment (fine-grained), hindering temporal
learning and video understanding. In this paper, we propose NOise Robust
Temporal Optimal …

arxiv captions computational cost cs.cv dependencies focus issue language long-term modeling solution studies temporal video videos