LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration

arXiv:2402.11550v1 Announce Type: new
Abstract: Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks. However, LLMs with long context windows have been notorious for their expensive training costs and high inference latency. Even the most advanced models such as GPT-4 and Claude2 often make mistakes when processing inputs of over $100k$ tokens, a phenomenon also known as \textit{lost in the middle}. In this paper, we propose \textsc{LongAgent}, a method based on multi-agent collaboration, …

128k context abstract advanced agent arxiv collaboration context context windows costs cs.ai cs.cl gpt gpt-4 inference inference latency language language models large language large language models latency llms multi-agent performance reasoning scaling tasks through training training costs type understanding windows