xLSTM: Extended Long Short-Term Memory

xLSTM is an architecture that combines the recurrency and constant memory requirement of LSTMs with the large-scale training of transformers and achieves impressive results.

Paper: https://arxiv.org/abs/2405.04517

Abstract:
In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the …

abstract architecture error ideas long short-term memory lstm memory results scale test training transformers xlstm