Accelerated Parameter-Free Stochastic Optimization

arXiv:2404.00666v1 Announce Type: new
Abstract: We propose a method that achieves near-optimal rates for smooth stochastic convex optimization and requires essentially no prior knowledge of problem parameters. This improves on prior work which requires knowing at least the initial distance to optimality d0. Our method, U-DoG, combines UniXGrad (Kavis et al., 2019) and DoG (Ivgi et al., 2023) with novel iterate stabilization techniques. It requires only loose bounds on d0 and the noise magnitude, provides high probability guarantees under sub-Gaussian …

abstract arxiv cs.lg dog free knowledge least math.oc near optimization parameters prior stochastic type work