SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series

arXiv:2403.15360v1 Announce Type: cross
Abstract: Transformers have widely adopted attention networks for sequence mixing and MLPs for channel mixing, playing a pivotal role in achieving breakthroughs across domains. However, recent literature highlights issues with attention networks, including low inductive bias and quadratic complexity concerning input sequence length. State Space Models (SSMs) like S4 and others (Hippo, Global Convolutions, liquid S4, LRU, Mega, and Mamba), have emerged to address the above issues to help handle longer sequence lengths. Mamba, while being …

architecture arxiv cs.cv cs.lg cs.sy eess.iv eess.sy mamba multivariate series simplified time series type vision