Claim Check-Worthiness Detection: How Well do LLMs Grasp Annotation Guidelines?

arXiv:2404.12174v1 Announce Type: new
Abstract: The increasing threat of disinformation calls for automating parts of the fact-checking pipeline. Identifying text segments requiring fact-checking is known as claim detection (CD) and claim check-worthiness detection (CW), the latter incorporating complex domain-specific criteria of worthiness and often framed as a ranking task. Zero- and few-shot LLM prompting is an attractive option for both tasks, as it bypasses the need for labeled datasets and allows verbalized claim and worthiness criteria to be directly used …

abstract annotation arxiv check claim cs.cl detection disinformation domain fact-checking guidelines llms pipeline ranking text threat type