A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning

arXiv:2306.07541v2 Announce Type: replace
Abstract: Offline reinforcement learning (RL) provides a promising solution to learning an agent fully relying on a data-driven paradigm. However, constrained by the limited quality of the offline dataset, its performance is often sub-optimal. Therefore, it is desired to further finetune the agent via extra online interactions before deployment. Unfortunately, offline-to-online RL can be challenging due to two main challenges: constrained exploratory behavior and state-action distribution shift. To this end, we propose a Simple Unified uNcertainty-Guided …

abstract agent arxiv cs.ai cs.lg data data-driven dataset extra framework interactions offline online reinforcement learning paradigm performance quality reinforcement reinforcement learning simple solution type uncertainty via