Sample-Efficient Constrained Reinforcement Learning with General Parameterization

arXiv:2405.10624v1 Announce Type: new
Abstract: We consider a constrained Markov Decision Problem (CMDP) where the goal of an agent is to maximize the expected discounted sum of rewards over an infinite horizon while ensuring that the expected discounted sum of costs exceeds a certain threshold. Building on the idea of momentum-based acceleration, we develop the Primal-Dual Accelerated Natural Policy Gradient (PD-ANPG) algorithm that guarantees an $\epsilon$ global optimality gap and $\epsilon$ constraint violation with $\mathcal{O}(\epsilon^{-3})$ sample complexity. This improves the …

abstract agent arxiv building costs cs.ai cs.lg decision general horizon markov reinforcement reinforcement learning sample sum threshold type while