Constrained Update Projection Approach to Safe Policy Optimization. (arXiv:2209.07089v1 [cs.LG])

Safe reinforcement learning (RL) studies problems where an intelligent agent
has to not only maximize reward but also avoid exploring unsafe areas. In this
study, we propose CUP, a novel policy optimization method based on Constrained
Update Projection framework that enjoys rigorous safety guarantee. Central to
our CUP development is the newly proposed surrogate functions along with the
performance bound. Compared to previous safe RL methods, CUP enjoys the
benefits of 1) CUP generalizes the surrogate functions to generalized advantage …

arxiv optimization policy projection