ToMBench: Benchmarking Theory of Mind in Large Language Models

arXiv:2402.15052v1 Announce Type: new
Abstract: Theory of Mind (ToM) is the cognitive capability to perceive and ascribe mental states to oneself and others. Recent research has sparked a debate over whether large language models (LLMs) exhibit a form of ToM. However, existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination, yielding inadequate assessments. To address this gap, we introduce ToMBench with three key characteristics: a systematic evaluation framework encompassing 8 tasks and 31 …

abstract arxiv benchmarking capability challenges cognitive cs.ai cs.cl form language language models large language large language models llms mind research theory theory of mind tom type