Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding

arXiv:2311.18773v2 Announce Type: replace
Abstract: Learning from videos is an emerging research area that enables robots to acquire skills from human demonstrations, such as procedural videos. To do this, video-language models must be able to obtain structured understandings, such as the temporal segmentation of a demonstration into sequences of actions and skills, and to generalize the understandings to novel domains. In pursuit of this goal, we introduce Spacewalk-18, a benchmark containing two tasks: (1) step recognition and (2) intra-video retrieval …

arxiv benchmark cs.cv form multimodal type understanding video video understanding