Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation

Stochastic Gradient Descent (SGD) with adaptive steps is now widely used for training deep neural networks. Most theoretical results assume access to unbiased gradient estimators, which is not the case in several recent deep learning and reinforcement learning applications that use Monte Carlo methods. This paper provides a comprehensive non-asymptotic analysis of SGD with biased gradients and adaptive steps for convex and non-convex smooth functions. Our study incorporates time-dependent bias and emphasizes the importance of controlling the bias and Mean …

analysis applications approximation case cs.lg deep learning gradient networks neural networks paper reinforcement reinforcement learning stat.ml stochastic training unbiased