Web: http://arxiv.org/abs/2006.07507

May 5, 2022, 1:12 a.m. | Keyi Chen, John Langford, Francesco Orabona

cs.LG updates on arXiv.org arxiv.org

Parameter-free stochastic gradient descent (PFSGD) algorithms do not require
setting learning rates while achieving optimal theoretical performance. In
practical applications, however, there remains an empirical gap between tuned
stochastic gradient descent (SGD) and PFSGD. In this paper, we close the
empirical gap with a new parameter-free algorithm based on continuous-time
Coin-Betting on truncated models. The new update is derived through the
solution of an Ordinary Differential Equation (ODE) and solved in a closed
form. We show empirically that this new …

arxiv free optimization stochastic

More from arxiv.org / cs.LG updates on arXiv.org

Data Analyst, Patagonia Action Works

@ Patagonia | Remote

Data & Insights Strategy & Innovation General Manager

@ Chevron Services Company, a division of Chevron U.S.A Inc. | Houston, TX

Faculty members in Research areas such as Bayesian and Spatial Statistics; Data Privacy and Security; AI/ML; NLP; Image and Video Data Analysis

@ Ahmedabad University | Ahmedabad, India

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC