Balancing Value Underestimation and Overestimation with Realistic Actor-Critic. (arXiv:2110.09712v6 [cs.LG] UPDATED)

Model-free deep reinforcement learning (RL) has been successfully applied to
challenging continuous control domains. However, poor sample efficiency
prevents these methods from being widely used in real-world domains. This paper
introduces a novel model-free algorithm, Realistic Actor-Critic(RAC), which can
be incorporated with any off-policy RL algorithms to improve sample efficiency.
RAC employs Universal Value Function Approximators (UVFA) to simultaneously
learn a policy family with the same neural network, each with different
trade-offs between underestimation and overestimation. To learn such policies, …

actor-critic arxiv value