Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Streaming Data. (arXiv:2109.07117v5 [cs.LG] UPDATED)

We consider the stochastic approximation problem in a streaming framework
where an objective is minimized through unbiased estimates of its gradients. In
this streaming framework, we consider time-varying data streams that must be
processed sequentially. Our methods are Stochastic Gradient (SG) based due to
their applicability and computational advantages. We provide a non-asymptotic
analysis of the convergence of various SG-based methods; this includes the
famous SG descent (a.k.a. Robbins-Monro algorithm), constant and time-varying
mini-batch SG methods, and their averaged estimates …

algorithms analysis approximation arxiv data lg stochastic streaming streaming data