Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning. (arXiv:2202.11566v1 [cs.LG])

Offline Reinforcement Learning (RL) aims to learn policies from previously
collected datasets without exploring the environment. Directly applying
off-policy algorithms to offline RL usually fails due to the extrapolation
error caused by the out-of-distribution (OOD) actions. Previous methods tackle
such problem by penalizing the Q-values of OOD actions or constraining the
trained policy to be close to the behavior policy. Nevertheless, such methods
typically prevent the generalization of value functions beyond the offline data
and also lack precise characterization of …

arxiv bootstrapping learning reinforcement reinforcement learning uncertainty