World Models via Policy-Guided Trajectory Diffusion

arXiv:2312.08533v3 Announce Type: replace
Abstract: World models are a powerful tool for developing intelligent agents. By predicting the outcome of a sequence of actions, world models enable policies to be optimised via on-policy reinforcement learning (RL) using synthetic data, i.e. in "in imagination". Existing world models are autoregressive in that they interleave predicting the next state with sampling the next action from the policy. Prediction error inevitably compounds as the trajectory length grows. In this work, we propose a novel …

abstract agents arxiv cs.ai cs.lg data diffusion imagination intelligent policies policy reinforcement reinforcement learning synthetic synthetic data tool trajectory type via world world models