WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment

arXiv:2402.12275v1 Announce Type: cross
Abstract: We give a model-based agent that builds a Python program representing its knowledge of the world based on its interactions with the environment. The world model tries to explain its interactions, while also being optimistic about what reward it can achieve. We do this by extending work on program synthesis via LLMs. We study our agent on gridworlds, finding our approach is more sample-efficient compared to deep RL, and more compute-efficient compared to ReAct-style agents.

abstract agent arxiv building code cs.ai cs.cl environment interactions knowledge llm python the environment type world world models writing