Neuro-Symbolic Video Search

arXiv:2403.11021v1 Announce Type: new
Abstract: The unprecedented surge in video data production in recent years necessitates efficient tools to extract meaningful frames from videos for downstream tasks. Long-term temporal reasoning is a key desideratum for frame retrieval systems. While state-of-the-art foundation models, like VideoLLaMA and ViCLIP, are proficient in short-term semantic understanding, they surprisingly fail at long-term reasoning across frames. A key reason for this failure is that they intertwine per-frame perception and temporal reasoning into a single deep network. …

abstract art arxiv cs.ai cs.cv data extract foundation key long-term neuro production reasoning retrieval search semantic state systems tasks temporal tools type understanding video video data videos