Detours for Navigating Instructional Videos

arXiv:2401.01823v2 Announce Type: replace
Abstract: We introduce the video detours problem for navigating instructional videos. Given a source video and a natural language query asking to alter the how-to video's current path of execution in a certain way, the goal is to find a related ''detour video'' that satisfies the requested alteration. To address this challenge, we propose VidDetours, a novel video-language approach that learns to retrieve the targeted temporal segments from a large repository of how-to's using video-and-text conditioned …

abstract arxiv cs.cv current how-to language natural natural language natural language query path query type video videos