Incorporating Exponential Smoothing into MLP: A Simple but Effective Sequence Model

arXiv:2403.17445v1 Announce Type: new
Abstract: Modeling long-range dependencies in sequential data is a crucial step in sequence learning. A recently developed model, the Structured State Space (S4), demonstrated significant effectiveness in modeling long-range sequences. However, It is unclear whether the success of S4 can be attributed to its intricate parameterization and HiPPO initialization or simply due to State Space Models (SSMs). To further investigate the potential of the deep SSMs, we start with exponential smoothing (ETS), a simple SSM, and …

abstract arxiv cs.ai cs.cl cs.lg data dependencies however mlp modeling sequence model simple space state success type