On the Parameterization and Initialization of Diagonal State Space Models. (arXiv:2206.11893v1 [cs.LG])

Web: http://arxiv.org/abs/2206.11893

State space models (SSM) have recently been shown to be very effective as a
deep learning layer as a promising alternative to sequence models such as RNNs,
CNNs, or Transformers. The first version to show this potential was the S4
model, which is particularly effective on tasks involving long-range
dependencies by using a prescribed state matrix called the HiPPO matrix. While
this has an interpretable mathematical mechanism for modeling long
dependencies, it introduces a custom representation and algorithm that can …

arxiv lg models on space state