Tracing Knowledge in Language Models Back to the Training Data. (arXiv:2205.11482v2 [cs.CL] UPDATED)

Neural language models (LMs) have been shown to memorize a great deal of
factual knowledge. But when an LM generates an assertion, it is often difficult
to determine where it learned this information and whether it is true. In this
paper, we introduce a new benchmark for fact tracing: tracing language models'
assertions back to the training examples that provided evidence for those
predictions. Prior work has suggested that dataset-level influence methods
might offer an effective framework for tracing predictions …

arxiv data knowledge language language models tracing training training data