The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning

arXiv:2402.12527v1 Announce Type: new
Abstract: Offline reinforcement learning aims to enable agents to be trained from pre-collected datasets, however, this comes with the added challenge of estimating the value of behavior not covered in the dataset. Model-based methods offer a solution by allowing agents to collect additional synthetic data via rollouts in a learned dynamics model. The prevailing theoretical understanding is that this can then be viewed as online reinforcement learning in an approximate dynamics model, and any remaining gap …

arxiv cs.ai cs.lg edge offline reinforcement reinforcement learning the edge type