LLM-Oriented Retrieval Tuner

arXiv:2403.01999v1 Announce Type: new
Abstract: Dense Retrieval (DR) is now considered as a promising tool to enhance the memorization capacity of Large Language Models (LLM) such as GPT3 and GPT-4 by incorporating external memories. However, due to the paradigm discrepancy between text generation of LLM and DR, it is still an open challenge to integrate the retrieval and generation tasks in a shared LLM. In this paper, we propose an efficient LLM-Oriented Retrieval Tuner, namely LMORT, which decouples DR capacity …

abstract arxiv capacity challenge cs.cl gpt gpt3 gpt-4 language language models large language large language models llm memories paradigm retrieval text text generation tool type