Do LLM Agents Have Regret? A Case Study in Online Learning and Games

arXiv:2403.16843v1 Announce Type: new
Abstract: Large language models (LLMs) have been increasingly employed for (interactive) decision-making, via the development of LLM-based autonomous agents. Despite their emerging successes, the performance of LLM agents in decision-making has not been fully investigated through quantitative metrics, especially in the multi-agent setting when they interact with each other, a typical scenario in real-world LLM-agent applications. To better understand the limits of LLM agents in these interactive environments, we propose to study their interactions in benchmark …

abstract agent agents arxiv autonomous autonomous agents case case study cs.ai cs.gt cs.lg decision development games interactive language language models large language large language models llm llms making metrics multi-agent online learning performance quantitative study through type via