Towards Provable Log Density Policy Gradient

arXiv:2403.01605v1 Announce Type: new
Abstract: Policy gradient methods are a vital ingredient behind the success of modern reinforcement learning. Modern policy gradient methods, although successful, introduce a residual error in gradient estimation. In this work, we argue that this residual term is significant and correcting for it could potentially improve sample-complexity of reinforcement learning methods. To that end, we propose log density gradient to estimate the policy gradient, which corrects for this residual error term. Log density gradient method computes …

abstract arxiv complexity cs.ai cs.lg error gradient modern policy reinforcement reinforcement learning residual sample stat.ml success type vital work