KEBench: A Benchmark on Knowledge Editing for Large Vision-Language Models

arXiv:2403.07350v1 Announce Type: new
Abstract: Currently, little research has been done on knowledge editing for Large Vision-Language Models (LVLMs). Editing LVLMs faces the challenge of effectively integrating diverse modalities (image and text) while ensuring coherent and contextually relevant modifications. An existing benchmark has three metrics (Reliability, Locality and Generality) to measure knowledge editing for LVLMs. However, the benchmark falls short in the quality of generated images used in evaluation and cannot assess whether models effectively utilize edited knowledge in relation …

abstract arxiv benchmark challenge cs.ai cs.cl cs.cv diverse editing image knowledge language language models metrics reliability research text type vision vision-language models