The Power of Resets in Online Reinforcement Learning

arXiv:2404.15417v1 Announce Type: new
Abstract: Simulators are a pervasive tool in reinforcement learning, but most existing algorithms cannot efficiently exploit simulator access -- particularly in high-dimensional domains that require general function approximation. We explore the power of simulators through online reinforcement learning with {local simulator access} (or, local planning), an RL protocol where the agent is allowed to reset to previously observed states and follow their dynamics during training. We use local simulator access to unlock new statistical guarantees that …

abstract access algorithms approximation arxiv cs.ai cs.lg domains exploit explore function general online reinforcement learning planning power protocol reinforcement reinforcement learning simulator stat.ml through tool type