Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning

arXiv:2402.10810v1 Announce Type: new
Abstract: We study the Constrained Convex Markov Decision Process (MDP), where the goal is to minimize a convex functional of the visitation measure, subject to a convex constraint. Designing algorithms for a constrained convex MDP faces several challenges, including (1) handling the large state space, (2) managing the exploration/exploitation tradeoff, and (3) solving the constrained optimization where the objective and the constraint are both nonlinear functions of the visitation measure. In this work, we present a …

abstract algorithms arxiv challenges cs.lg decision designing functional markov math.oc optimization policy primal process reinforcement reinforcement learning space state stat.ml study type