ST-MambaSync: The Confluence of Mamba Structure and Spatio-Temporal Transformers for Precipitous Traffic Prediction

arXiv:2404.15899v1 Announce Type: new
Abstract: Balancing accuracy with computational efficiency is paramount in machine learning, particularly when dealing with high-dimensional data, such as spatial-temporal datasets. This study introduces ST-MambaSync, an innovative framework that integrates a streamlined attention layer with a simplified state-space layer. The model achieves competitive accuracy in spatial-temporal prediction tasks. We delve into the relationship between attention mechanisms and the Mamba component, revealing that Mamba functions akin to attention within a residual network structure. This comparative analysis underpins …

abstract accuracy arxiv attention computational confluence cs.ai cs.lg data datasets efficiency framework layer machine machine learning mamba prediction simplified space spatial state study temporal traffic transformers type