Penalized Proximal Policy Optimization for Safe Reinforcement Learning. (arXiv:2205.11814v1 [cs.LG])

Safe reinforcement learning aims to learn the optimal policy while satisfying
safety constraints, which is essential in real-world applications. However,
current algorithms still struggle for efficient policy updates with hard
constraint satisfaction. In this paper, we propose Penalized Proximal Policy
Optimization (P3O), which solves the cumbersome constrained policy iteration
via a single minimization of an equivalent unconstrained problem. Specifically,
P3O utilizes a simple-yet-effective penalty function to eliminate cost
constraints and removes the trust-region constraint by the clipped surrogate
objective. We …

arxiv learning optimization policy reinforcement reinforcement learning