Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?

arXiv:2406.13121v1 Announce Type: new
Abstract: Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. Leveraging LCLMs' ability to natively ingest and process entire corpora of information offers numerous advantages. It enhances user-friendliness by eliminating the need for specialized knowledge of tools, provides robust end-to-end modeling that minimizes cascading errors in complex pipelines, and allows for the application of sophisticated prompting techniques across the entire system. To …

abstract advantages arxiv context cs.ai cs.cl cs.ir databases information language language models potential process rag retrieval sql systems tasks tools type