Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning

arXiv:2310.05723v2 Announce Type: replace
Abstract: Offline pretraining with a static dataset followed by online fine-tuning (offline-to-online, or OtO) is a paradigm well matched to a real-world RL deployment process. In this scenario, we aim to find the best-performing policy within a limited budget of online interactions. Previous work in the OtO setting has focused on correcting for bias introduced by the policy-constraint mechanisms of offline RL algorithms. Such constraints keep the learned policy close to the behavior policy that collected …

abstract aim arxiv budget cs.lg dataset deployment distribution fine-tuning interactions offline online reinforcement learning paradigm planning policy pretraining process reinforcement reinforcement learning type work world