Volume-Preserving Transformers for Learning Time Series Data with Structure

arXiv:2312.11166v2 Announce Type: replace-cross
Abstract: Two of the many trends in neural network research of the past few years have been (i) the learning of dynamical systems, especially with recurrent neural networks such as long short-term memory networks (LSTMs) and (ii) the introduction of transformer neural networks for natural language processing (NLP) tasks. Both of these trends have created enormous amounts of traction, particularly the second one: transformer networks now dominate the field of NLP. Even though some work has …

abstract arxiv cs.lg cs.na data introduction language language processing long short-term memory math.na memory natural natural language natural language processing network networks neural network neural networks processing recurrent neural networks research series systems time series transformer transformers trends type