Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup

arXiv:2403.00820v1 Announce Type: cross
Abstract: Retrieval Augmented Generation (RAG) systems have seen huge popularity in augmenting Large-Language Model (LLM) outputs with domain specific and time sensitive data. Very recently a shift is happening from simple RAG setups that query a vector database for additional information with every user input to more sophisticated forms of RAG. However, different concrete approaches compete on mostly anecdotal evidence at the moment. In this paper we present a rigorous dataset creation and evaluation workflow to …

abstract agent arxiv cs.cl cs.ir data database dataset domain evaluation every information language language model llm query rag retrieval retrieval augmented generation setup shift simple systems type vector vector database