Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

arXiv:2303.10599v2 Announce Type: replace
Abstract: Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since …

abstract analysis applications arxiv convergence entropy estimator functions gradient however machine machine learning math.oc mcmc stat.ml stochastic type unbiased understanding variants