When should we prefer Decision Transformers for Offline Reinforcement Learning?

arXiv:2305.14550v3 Announce Type: replace
Abstract: Offline reinforcement learning (RL) allows agents to learn effective, return-maximizing policies from a static dataset. Three popular algorithms for offline RL are Conservative Q-Learning (CQL), Behavior Cloning (BC), and Decision Transformer (DT), from the class of Q-Learning, Imitation Learning, and Sequence Modeling respectively. A key open question is: which algorithm is preferred under what conditions? We study this question empirically by exploring the performance of these algorithms across the commonly used D4RL and Robomimic benchmarks. …

abstract agents algorithms arxiv behavior class cloning cs.ai cs.lg dataset decision imitation learning key learn modeling offline popular q-learning reinforcement reinforcement learning transformer transformers type