Robust Lagrangian and Adversarial Policy Gradient for Robust Constrained Markov Decision Processes

arXiv:2308.11267v2 Announce Type: replace
Abstract: The robust constrained Markov decision process (RCMDP) is a recent task-modelling framework for reinforcement learning that incorporates behavioural constraints and that provides robustness to errors in the transition dynamics model through the use of an uncertainty set. Simulating RCMDPs requires computing the worst-case dynamics based on value estimates for each state, an approach which has previously been used in the Robust Constrained Policy Gradient (RCPG). Highlighting potential downsides of RCPG such as not robustifying the …

abstract adversarial arxiv case computing constraints cs.ai cs.lg cs.ne decision dynamics errors framework gradient markov modelling policy process processes reinforcement reinforcement learning robust robustness set through transition type uncertainty