A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics. (arXiv:2207.09304v1 [math.PR])

We establish a sharp uniform-in-time error estimate for the Stochastic
Gradient Langevin Dynamics (SGLD), which is a popular sampling algorithm. Under
mild assumptions, we obtain a uniform-in-time $O(\eta^2)$ bound for the
KL-divergence between the SGLD iteration and the Langevin diffusion, where
$\eta$ is the step size (or learning rate). Our analysis is also valid for
varying step sizes. Based on this, we are able to obtain an $O(\eta)$ bound for
the distance between the SGLD iteration and the invariant distribution …

arxiv dynamics error gradient math pr stochastic time uniform