Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning

arXiv:2403.09621v1 Announce Type: new
Abstract: Distributionally robust offline reinforcement learning (RL), which seeks robust policy training against environment perturbation by modeling dynamics uncertainty, calls for function approximations when facing large state-action spaces. However, the consideration of dynamics uncertainty introduces essential nonlinearity and computational burden, posing unique challenges for analyzing and practically employing function approximation. Focusing on a basic setting where the nominal model and perturbed models are linearly parameterized, we propose minimax optimal and computationally efficient algorithms realizing function approximation …

abstract algorithms arxiv challenges computational cs.ai cs.lg dynamics environment function however minimax modeling offline policy reinforcement reinforcement learning robust spaces state stat.ml training type uncertainty