Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong

arXiv:2310.12558v2 Announce Type: replace
Abstract: Large Language Models (LLMs) are increasingly used for accessing information on the web. Their truthfulness and factuality are thus of great interest. To help users make the right decisions about the information they get, LLMs should not only provide information but also help users fact-check it. Our experiments with 80 crowdworkers compare language models with search engines (information retrieval systems) at facilitating fact-checking. We prompt LLMs to validate a given claim and provide corresponding explanations. …

abstract arxiv cs.cl cs.hc decisions humans information language language models large language large language models llms the information type verify web