Scalable Model Editing via Customized Expert Networks

arXiv:2404.02699v1 Announce Type: new
Abstract: Addressing the issue of hallucinations and outdated knowledge in large language models is critical for their reliable application. Model Editing presents a promising avenue for mitigating these challenges in a cost-effective manner. However, existing methods often suffer from unsatisfactory generalization and unintended effects on unrelated samples. To overcome these limitations, we introduce a novel approach: Scalable Model Editing via Customized Expert Networks (SCEN), which is a two-stage continuous training paradigm. Specifically, in the first stage, …

abstract application arxiv challenges cost cs.cl editing effects expert hallucinations however issue knowledge language language models large language large language models networks samples scalable type via