SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding

arXiv:2404.01174v1 Announce Type: new
Abstract: Temporal video grounding (TVG) is a critical task in video content understanding. Despite significant advancements, existing methods often limit in capturing the fine-grained relationships between multimodal inputs and the high computational costs with processing long video sequences. To address these limitations, we introduce a novel SpikeMba: multi-modal spiking saliency mamba for temporal video grounding. In our work, we integrate the Spiking Neural Networks (SNNs) and state space models (SSMs) to capture the fine-grained relationships of …

abstract arxiv computational costs cs.cv cs.mm fine-grained inputs limitations mamba modal multi-modal multimodal novel processing relationships temporal type understanding video