Adaptive Rollout Length for Model-Based RL Using Model-Free Deep RL. (arXiv:2206.02380v2 [cs.LG] UPDATED)

Model-based reinforcement learning promises to learn an optimal policy from
fewer interactions with the environment compared to model-free reinforcement
learning by learning an intermediate model of the environment in order to
predict future interactions. When predicting a sequence of interactions, the
rollout length, which limits the prediction horizon, is a critical
hyperparameter as accuracy of the predictions diminishes in the regions that
are further away from real experience. As a result, with a longer rollout
length, an overall worse policy …

arxiv deep rl free lg rl