Learning method for S4 with Diagonal State Space Layers using Balanced Truncation

arXiv:2402.15993v1 Announce Type: new
Abstract: We introduce a novel learning method for Structured State Space Sequence (S4) models incorporating Diagonal State Space (DSS) layers, tailored for processing long-sequence data in edge intelligence applications, including sensor data analysis and real-time analytics. This method utilizes the balanced truncation technique, prevalent in control theory, applied specifically to DSS layers to reduce computational costs during inference. By leveraging parameters from the reduced model, we refine the initialization process of S4 models, outperforming the widely …

abstract analysis analytics applications arxiv cs.lg data data analysis edge edge intelligence intelligence novel processing real-time sensor space state type