TAXI: Evaluating Categorical Knowledge Editing for Language Models

arXiv:2404.15004v1 Announce Type: new
Abstract: Humans rarely learn one fact in isolation. Instead, learning a new fact induces knowledge of other facts about the world. For example, in learning a korat is a type of cat, you also infer it is a mammal and has claws, ensuring your model of the world is consistent. Knowledge editing aims to inject new facts into language models to improve their factuality, but current benchmarks fail to evaluate consistency, which is critical to ensure …

arxiv categorical cs.cl editing knowledge language language models taxi type