[Research] xLSTM: Extended Long Short-Term Memory

Abstract:

In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the Transformer technology with parallelizable self-attention at its core marked the dawn of a new era, outpacing LSTMs at scale. We now raise a simple question: How …

abstract contributed deep learning error however ideas language language models large language large language models llms long short-term memory lstm machinelearning memory research stories success success stories test