How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO

arXiv:2404.13957v1 Announce Type: new
Abstract: The role-play ability of Large Language Models (LLMs) has emerged as a popular research direction. However, existing studies focus on imitating well-known public figures or fictional characters, overlooking the potential for simulating ordinary individuals. Such an oversight limits the potential for advancements in digital human clones and non-player characters in video games. To bridge this gap, we introduce ECHO, an evaluative framework inspired by the Turing test. This framework engages the acquaintances of the target …

ai chatbots arxiv chatbots cs.cl echo llms role type