Variance Reduction and Low Sample Complexity in Stochastic Optimization via Proximal Point Method

arXiv:2402.08992v1 Announce Type: cross
Abstract: This paper proposes a stochastic proximal point method to solve a stochastic convex composite optimization problem. High probability results in stochastic optimization typically hinge on restrictive assumptions on the stochastic gradient noise, for example, sub-Gaussian distributions. Assuming only weak conditions such as bounded variance of the stochastic gradient, this paper establishes a low sample complexity to obtain a high probability guarantee on the convergence of the proposed method. Additionally, a notable aspect of this work …

abstract arxiv assumptions complexity cs.lg example gradient hinge low math.oc noise optimization paper probability restrictive sample solve stat.ml stochastic type variance via