Safe Exploration Using Bayesian World Models and Log-Barrier Optimization

arXiv:2405.05890v1 Announce Type: new
Abstract: A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process. In this work, we propose CERL, a new method for solving constrained Markov decision processes while keeping the policy safe during learning. Our method leverages Bayesian world models and suggests policies that are pessimistic w.r.t. the model's epistemic uncertainty. This makes CERL robust towards model inaccuracies and leads to safe exploration during learning. In our …

abstract arxiv bayesian challenge cs.ai cs.lg decision exploration major markov optimization policy process processes reinforcement reinforcement learning safe safety tasks type while work world world models