Video-Guided Curriculum Learning for Spoken Video Grounding. (arXiv:2209.00277v1 [cs.CV])

In this paper, we introduce a new task, spoken video grounding (SVG), which
aims to localize the desired video fragments from spoken language descriptions.
Compared with using text, employing audio requires the model to directly
exploit the useful phonemes and syllables related to the video from raw speech.
Moreover, we randomly add environmental noises to this speech audio, further
increasing the difficulty of this task and better simulating real applications.
To rectify the discriminative phonemes and extract video-related information
from …

arxiv curriculum curriculum learning learning video