ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints

arXiv:2302.00808v3 Announce Type: replace
Abstract: Reinforcement Learning (RL) for constrained MDPs (CMDPs) is an increasingly important problem for various applications. Often, the average criterion is more suitable than the discounted criterion. Yet, RL for average-CMDPs (ACMDPs) remains a challenging problem. Algorithms designed for discounted constrained RL problems often do not perform well for the average CMDP setting. In this paper, we introduce a new policy optimization with function approximation algorithm for constrained MDPs with the average criterion. The Average-Constrained Policy …

abstract algorithm algorithms applications arxiv constraints criterion cs.ai cs.lg optimization policy reinforcement reinforcement learning type