The Generalization Gap in Offline Reinforcement Learning

arXiv:2312.05742v2 Announce Type: replace
Abstract: Despite recent progress in offline learning, these methods are still trained and tested on the same environment. In this paper, we compare the generalization abilities of widely used online and offline learning methods such as online reinforcement learning (RL), offline RL, sequence modeling, and behavioral cloning. Our experiments show that offline learning algorithms perform worse on new environments than online learning ones. We also introduce the first benchmark for evaluating generalization in offline learning, collecting …

abstract arxiv cloning cs.ai cs.lg environment gap modeling offline online reinforcement learning paper progress reinforcement reinforcement learning type