Can Language Models Recognize Convincing Arguments?

arXiv:2404.00750v1 Announce Type: new
Abstract: The remarkable and ever-increasing capabilities of Large Language Models (LLMs) have raised concerns about their potential misuse for creating personalized, convincing misinformation and propaganda. To gain insights into LLMs' persuasive capabilities without directly engaging in experimentation with humans, we propose studying their performance on the related task of detecting convincing arguments. We extend a dataset by Durmus & Cardie (2018) with debates, votes, and user traits and propose tasks measuring LLMs' ability to (1) distinguish …

abstract arxiv capabilities concerns cs.cl cs.cy experimentation humans insights language language models large language large language models llms misinformation misuse performance personalized propaganda studying type