DOMAIN: MilDly COnservative Model-BAsed OfflINe Reinforcement Learning

arXiv:2309.08925v2 Announce Type: replace
Abstract: Model-based reinforcement learning (RL), which learns environment model from offline dataset and generates more out-of-distribution model data, has become an effective approach to the problem of distribution shift in offline RL. Due to the gap between the learned and actual environment, conservatism should be incorporated into the algorithm to balance accurate offline data and imprecise model data. The conservatism of current algorithms mostly relies on model uncertainty estimation. However, uncertainty estimation is unreliable and leads …

abstract algorithm arxiv become cs.ai cs.lg data dataset distribution domain environment gap offline reinforcement reinforcement learning shift the algorithm type