Can large language models explore in-context?

arXiv:2403.15371v1 Announce Type: new
Abstract: We investigate the extent to which contemporary Large Language Models (LLMs) can engage in exploration, a core capability in reinforcement learning and decision making. We focus on native performance of existing LLMs, without training interventions. We deploy LLMs as agents in simple multi-armed bandit environments, specifying the environment description and interaction history entirely in-context, i.e., within the LLM prompt. We experiment with GPT-3.5, GPT-4, and Llama2, using a variety of prompt designs, and find that …

abstract agents arxiv capability context core cs.ai cs.cl cs.lg decision decision making deploy environment environments exploration explore focus language language models large language large language models llms making performance reinforcement reinforcement learning simple the environment training type