Constrained Variational Policy Optimization for Safe Reinforcement Learning. (arXiv:2201.11927v1 [cs.LG])

Web: http://arxiv.org/abs/2201.11927

Safe reinforcement learning (RL) aims to learn policies that satisfy certain
constraints before deploying to safety-critical applications. Primal-dual as a
prevalent constrained optimization framework suffers from instability issues
and lacks optimality guarantees. This paper overcomes the issues from a novel
probabilistic inference perspective and proposes an Expectation-Maximization
style approach to learn safe policy. We show that the safe RL problem can be
decomposed to 1) a convex optimization phase with a non-parametric variational
distribution and 2) a supervised learning phase. …

arxiv learning optimization policy reinforcement learning