On-Policy Model Errors in Reinforcement Learning. (arXiv:2110.07985v2 [cs.LG] UPDATED)

Model-free reinforcement learning algorithms can compute policy gradients
given sampled environment transitions, but require large amounts of data. In
contrast, model-based methods can use the learned model to generate new data,
but model errors and bias can render learning unstable or suboptimal. In this
paper, we present a novel method that combines real-world data and a learned
model in order to get the best of both worlds. The core idea is to exploit the
real-world data for on-policy predictions and …

arxiv errors learning policy reinforcement reinforcement learning