Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Reddit

arXiv:2404.01147v1 Announce Type: cross
Abstract: Large language models (LLMs) have been shown to be proficient in correctly answering questions in the context of online discourse. However, the study of using LLMs to model human-like answers to fact-driven social media questions is still under-explored. In this work, we investigate how LLMs model the wide variety of human answers to fact-driven questions posed on several topic-specific Reddit communities, or subreddits. We collect and release a dataset of 409 fact-driven questions and 7,534 …

abstract arxiv case case study context cs.cl cs.lg discourse however human human-like language language models large language large language models llms media questions reddit social social media study type work