COMBO: Conservative Offline Model-Based Policy Optimization. (arXiv:2102.08363v2 [cs.LG] UPDATED)

Model-based algorithms, which learn a dynamics model from logged experience
and perform some sort of pessimistic planning under the learned model, have
emerged as a promising paradigm for offline reinforcement learning (offline
RL). However, practical variants of such model-based algorithms rely on
explicit uncertainty quantification for incorporating pessimism. Uncertainty
estimation with complex models, such as deep neural networks, can be difficult
and unreliable. We overcome this limitation by developing a new model-based
offline RL algorithm, COMBO, that regularizes the value …

arxiv optimization policy