Closing the gap between SVRG and TD-SVRG with Gradient Splitting

arXiv:2211.16237v3 Announce Type: replace
Abstract: Temporal difference (TD) learning is a policy evaluation in reinforcement learning whose performance can be enhanced by variance reduction methods. Recently, multiple works have sought to fuse TD learning with Stochastic Variance Reduced Gradient (SVRG) method to achieve a geometric rate of convergence. However, the resulting convergence rate is significantly weaker than what is achieved by SVRG in the setting of convex optimization. In this work we utilize a recent interpretation of TD-learning as the …

abstract arxiv convergence cs.lg difference evaluation gap gradient however multiple performance policy rate reinforcement reinforcement learning stochastic temporal type variance