The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model

arXiv:2305.16589v2 Announce Type: replace
Abstract: This paper investigates model robustness in reinforcement learning (RL) to reduce the sim-to-real gap in practice. We adopt the framework of distributionally robust Markov decision processes (RMDPs), aimed at learning a policy that optimizes the worst-case performance when the deployed environment falls within a prescribed uncertainty set around the nominal MDP. Despite recent efforts, the sample complexity of RMDPs remained mostly unsettled regardless of the uncertainty set in use. It was unclear if distributional robustness …

abstract arxiv case cs.it cs.lg decision environment framework gap generative markov math.it math.st model robustness paper performance policy practice price processes reduce reinforcement reinforcement learning robust robustness sim stat.th type