Offline RL Policies Should be Trained to be Adaptive. (arXiv:2207.02200v1 [cs.LG])

Offline RL algorithms must account for the fact that the dataset they are
provided may leave many facets of the environment unknown. The most common way
to approach this challenge is to employ pessimistic or conservative methods,
which avoid behaviors that are too dissimilar from those in the training
dataset. However, relying exclusively on conservatism has drawbacks:
performance is sensitive to the exact degree of conservatism, and conservative
objectives can recover highly suboptimal policies. In this work, we propose
that …

arxiv lg rl