Variance Reduction for Policy-Gradient Methods via Empirical Variance Minimization. (arXiv:2206.06827v2 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2206.06827

Policy-gradient methods in Reinforcement Learning(RL) are very universal and
widely applied in practice but their performance suffers from the high variance
of the gradient estimate. Several procedures were proposed to reduce it
including actor-critic(AC) and advantage actor-critic(A2C) methods. Recently
the approaches have got new perspective due to the introduction of Deep RL:
both new control variates(CV) and new sub-sampling procedures became available
in the setting of complex models like neural networks. The vital part of
CV-based methods is the goal …

arxiv gradient lg policy policy-gradient variance