Unsupervised Pretraining for Fact Verification by Language Model Distillation

arXiv:2309.16540v3 Announce Type: replace-cross
Abstract: Fact verification aims to verify a claim using evidence from a trustworthy knowledge base. To address this challenge, algorithms must produce features for every claim that are both semantically meaningful, and compact enough to find a semantic alignment with the source information. In contrast to previous work, which tackled the alignment problem by learning over annotated corpora of claims and their corresponding labels, we propose SFAVEL (Self-supervised Fact Verification via Language Model Distillation), a novel …

abstract algorithms alignment arxiv challenge claim contrast cs.cl cs.lg distillation every evidence features information knowledge knowledge base language language model model distillation pretraining semantic stat.ml trustworthy type unsupervised verification verify