Web: http://arxiv.org/abs/2206.11872

June 24, 2022, 1:10 a.m. | Jun-Kun Wang, Chi-Heng Lin, Andre Wibisono, Bin Hu

cs.LG updates on arXiv.org arxiv.org

Heavy Ball (HB) nowadays is one of the most popular momentum methods in
non-convex optimization. It has been widely observed that incorporating the
Heavy Ball dynamic in gradient-based methods accelerates the training process
of modern machine learning models. However, the progress on establishing its
theoretical foundation of acceleration is apparently far behind its empirical
success. Existing provable acceleration results are of the quadratic or
close-to-quadratic functions, as the current techniques of showing HB's
acceleration are limited to the case when …

arxiv math

More from arxiv.org / cs.LG updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY