Realistic Actor-Critic: A Framework for Balance Between Value Overestimation and Underestimation. (arXiv:2110.09712v5 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2110.09712

This paper proposes a reinforcement learning framework to enhance the
exploration-exploitation trade-off by learning a range of policies concerning
various confidence bounds. The underestimated values provide stable updates but
suffer from inefficient exploration behaviors. On the other hand, overestimated
values can help the agent escape local optima, but it might cause
over-exploration on low-value areas and function approximation errors
accumulation. Algorithms have been proposed to mitigate the above
contradiction. However, we lack an understanding of how the value bias impact …

actor-critic arxiv framework lg value