Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding

arXiv:2403.09626v1 Announce Type: new
Abstract: Understanding videos is one of the fundamental directions in computer vision research, with extensive efforts dedicated to exploring various architectures such as RNN, 3D CNN, and Transformers. The newly proposed architecture of state space model, e.g., Mamba, shows promising traits to extend its success in long sequence modeling to video modeling. To assess whether Mamba can be a viable alternative to Transformers in the video understanding domain, in this work, we conduct a comprehensive set …

arxiv cs.cv mamba space state type understanding video video understanding