Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works

arXiv:2404.12726v1 Announce Type: new
Abstract: Large language models (LLMs) have demonstrated impressive performance and spurred numerous AI applications, in which role-playing agents (RPAs) are particularly popular, especially for fictional characters. The prerequisite for these RPAs lies in the capability of LLMs to understand characters from fictional works. Previous efforts have evaluated this capability via basic classification tasks or characteristic imitation, failing to capture the nuanced character understanding with LLMs. In this paper, we propose evaluating LLMs' character understanding capability via …

arxiv cs.cl language language models large language large language models profiling type understanding via