Self-Extend LLM: Upgrade your context length

Self-Extend LLM: When LLMs encounter text sequences during inference - exceeding the length of their pre-training context window, we are faced with out-of-distribution (O.O.D) issues related to positional encoding.

Neural networks (NNs) and in particular LLMs are susceptible to unpredictable behaviors when dealing with O.O.D inputs. We analyse a new solution, to increase the context length of LLM during inference!

Introducing grouped self-attention, that extends the classical self-attention of transformers outside of their pre-trained context length!

All rights w/ authors: …

context context window distribution encoding inference inputs llm llms networks neural networks nns positional encoding pre-training solution text training upgrade