FASTTRACK: Fast and Accurate Fact Tracing for LLMs

arXiv:2404.15157v1 Announce Type: new
Abstract: Fact tracing seeks to identify specific training examples that serve as the knowledge source for a given query. Existing approaches to fact tracing rely on assessing the similarity between each training sample and the query along a certain dimension, such as lexical similarity, gradient, or embedding space. However, these methods fall short of effectively distinguishing between samples that are merely relevant and those that actually provide supportive evidence for the information sought by the query. …

abstract arxiv cs.ai cs.cl embedding examples gradient identify knowledge llms query sample serve space tracing training type