MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory

arXiv:2404.11672v1 Announce Type: new
Abstract: While current large language models (LLMs) demonstrate some capabilities in knowledge-intensive tasks, they are limited by relying on their parameters as an implicit storage mechanism. As a result, they struggle with infrequent knowledge and temporal degradation. In addition, the uninterpretable nature of parametric memorization makes it challenging to understand and prevent hallucination. Parametric memory pools and model editing are only partial solutions. Retrieval Augmented Generation (RAG) $\unicode{x2013}$ though non-parametric $\unicode{x2013}$ has its own limitations: it …

abstract arxiv capabilities cs.cl current finetuning knowledge language language models large language large language models llms memory nature parameters parametric storage struggle tasks temporal type