Shocking: 2/3 of LLMs fail at 2K context length

Context length is a limiting factor for Large Language Models, especially for RAG systems. But 2/3 of current LLMs fail at 2K token length, in Jan 2024! Is RAG-induced multi-placement a solution?

My test prompt is with 5272 characters too long to upload to Youtube.
I use a prompt of 741 words, corresponding to 1254 tokens (of my adapted tokenizer).
Every tested models applies its specific tokenization.

It is amazing, that open source LLM outperform even huge commercial LLMs.

By …

characters context current jan language language models large language large language models llms placement prompt rag solution systems test token youtube