DeepEdit: Knowledge Editing as Decoding with Constraints

arXiv:2401.10471v2 Announce Type: replace
Abstract: We propose a new perspective of knowledge editing (KE) for large language models (LLMs) that treats it as a constrained decoding problem. We design decoding constraints to regulate LLMs, ensuring coherence between reasoning steps when incorporating new knowledge. To enforce these constraints, we utilize a depth-first search to adaptively substitute new knowledge for the LLMs' original reasoning steps, greedily seeking the optimal path of multi-hop reasoning with new knowledge. From this vantage, we propose DEEPEDIT: …

abstract arxiv constraints cs.ai cs.cl decoding design editing knowledge language language models large language large language models llms perspective reasoning search type