Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding

arXiv:2401.00901v2 Announce Type: replace
Abstract: Video grounding aims to localize a spatio-temporal section in a video corresponding to an input text query. This paper addresses a critical limitation in current video grounding methodologies by introducing an Open-Vocabulary Spatio-Temporal Video Grounding task. Unlike prevalent closed-set approaches that struggle with open-vocabulary scenarios due to limited training data and predefined vocabularies, our model leverages pre-trained representations from foundational spatial grounding models. This empowers it to effectively bridge the semantic gap between natural language …

abstract arxiv cs.cv current paper query set struggle temporal text training type video