RvS: What is Essential for Offline RL via Supervised Learning?. (arXiv:2112.10751v2 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2112.10751

Recent work has shown that supervised learning alone, without temporal
difference (TD) learning, can be remarkably effective for offline RL. When does
this hold true, and which algorithmic components are necessary? Through
extensive experiments, we boil supervised learning for offline RL down to its
essential elements. In every environment suite we consider, simply maximizing
likelihood with a two-layer feedforward MLP is competitive with
state-of-the-art results of substantially more complex methods based on TD
learning or sequence modeling with Transformers. Carefully …

arxiv learning rl supervised learning