Deep autoregressive density nets vs neural ensembles for model-based offline reinforcement learning

We consider the problem of offline reinforcement learning where only a set of system transitions is made available for policy optimization. Following recent advances in the field, we consider a model-based reinforcement learning algorithm that infers the system dynamics from the available data and performs policy optimization on imaginary model rollouts. This approach is vulnerable to exploiting model errors which can lead to catastrophic failures on the real system. The standard solution is to rely on ensembles for uncertainty heuristics …

advances algorithm cs.lg data dynamics offline optimization policy reinforcement reinforcement learning set stat.ml transitions