Policy Bifurcation in Safe Reinforcement Learning

arXiv:2403.12847v1 Announce Type: new
Abstract: Safe reinforcement learning (RL) offers advanced solutions to constrained optimal control problems. Existing studies in safe RL implicitly assume continuity in policy functions, where policies map states to actions in a smooth, uninterrupted manner; however, our research finds that in some scenarios, the feasible policy should be discontinuous or multi-valued, interpolating between discontinuous local optima can inevitably lead to constraint violations. We are the first to identify the generating mechanism of such a phenomenon, and …

abstract advanced arxiv continuity control cs.lg functions however map policy reinforcement reinforcement learning research solutions studies type