Explicit Lipschitz Value Estimation Enhances Policy Robustness Against Perturbation

arXiv:2404.13879v1 Announce Type: new
Abstract: In robotic control tasks, policies trained by reinforcement learning (RL) in simulation often experience a performance drop when deployed on physical hardware, due to modeling error, measurement error, and unpredictable perturbations in the real world. Robust RL methods account for this issue by approximating a worst-case value function during training, but they can be sensitive to approximation errors in the value function and its gradient before training is complete. In this paper, we hypothesize that …

abstract arxiv case control cs.lg error experience hardware issue measurement modeling performance policies policy reinforcement reinforcement learning robotic robust robustness simulation tasks type value world