Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning

arXiv:2405.03279v1 Announce Type: new
Abstract: Model editing aims to correct outdated or erroneous knowledge in large language models (LLMs) without the need for costly retraining. Lifelong model editing is the most challenging task that caters to the continuous editing requirements of LLMs. Prior works primarily focus on single or batch editing; nevertheless, these methods fall short in lifelong editing scenarios due to catastrophic knowledge forgetting and the degradation of model performance. Although retrieval-based methods alleviate these issues, they are impeded …

abstract arxiv continuous cs.cl editing focus knowledge language language models large language large language models llms prior prompt prompt learning requirements retraining retrieval retrieval-augmented type