NEW: Unlimited Token Length for LLMs by Microsoft (LongNet explained)

New research by Microsoft: LongNet w/ 1billion token length for your LLM.
A new Super Intelligence as Captain Picard calls it? Or just another mathematical approximation and therefore a simplification of self-attention in Transformers by applying a simple Dilated Attention?

LONGNET: Scaling Transformers to
1,000,000,000 Tokens
https://arxiv.org/pdf/2307.02486.pdf

#ai
#largelanguagemodels
#promptengineering

approximation attention explained intelligence largelanguagemodels llm llms microsoft picard promptengineering research scaling self-attention simple token tokens transformers