Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learning

arXiv:2310.03718v2 Announce Type: replace
Abstract: Safe reinforcement learning (RL) focuses on training reward-maximizing agents subject to pre-defined safety constraints. Yet, learning versatile safe policies that can adapt to varying safety constraint requirements during deployment without retraining remains a largely unexplored and challenging area. In this work, we formulate the versatile safe RL problem and consider two primary requirements: training efficiency and zero-shot adaptation capability. To address them, we introduce the Conditioned Constrained Policy Optimization (CCPO) framework, consisting of two key …

abstract adapt agents arxiv constraints cs.ai cs.lg deployment optimization policies policy reinforcement reinforcement learning requirements retraining safe safety training type work