Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RL. (arXiv:2206.00695v1 [cs.LG])

We introduce an offline reinforcement learning (RL) algorithm that explicitly
clones a behavior policy to constrain value learning. In offline RL, it is
often important to prevent a policy from selecting unobserved actions, since
the consequence of these actions cannot be presumed without additional
information about the environment. One straightforward way to implement such a
constraint is to explicitly model a given data distribution via behavior
cloning and directly force a policy not to select uncertain actions. However,
many offline …

arxiv rl