A Natural Extension To Online Algorithms For Hybrid RL With Limited Coverage

arXiv:2403.09701v1 Announce Type: new
Abstract: Hybrid Reinforcement Learning (RL), leveraging both online and offline data, has garnered recent interest, yet research on its provable benefits remains sparse. Additionally, many existing hybrid RL algorithms (Song et al., 2023; Nakamoto et al., 2023; Amortila et al., 2024) impose coverage assumptions on the offline dataset, but we show that this is unnecessary. A well-designed online algorithm should "fill in the gaps" in the offline dataset, exploring states and actions that the behavior policy …

abstract algorithms arxiv assumptions benefits coverage cs.lg data extension hybrid natural offline reinforcement reinforcement learning research song stat.ml type