Web: http://arxiv.org/abs/2204.11326

May 12, 2022, 1:11 a.m. | Chao Ma, Daniel Kunin, Lei Wu, Lexing Ying

cs.LG updates on arXiv.org arxiv.org

Local quadratic approximation has been extensively used to study the
optimization of neural network loss functions around the minimum. Though, it
usually holds in a very small neighborhood of the minimum, and cannot explain
many phenomena observed during the optimization process. In this work, we study
the structure of neural network loss functions and its implication on
optimization in a region beyond the reach of good quadratic approximation.
Numerically, we observe that neural network loss functions possesses a
multiscale structure, …

arxiv loss network neural neural network on optimization

More from arxiv.org / cs.LG updates on arXiv.org

Predictive Ecology Postdoctoral Fellow

@ Lawrence Berkeley National Lab | Berkeley, CA

Data Analyst, Patagonia Action Works

@ Patagonia | Remote

Data & Insights Strategy & Innovation General Manager

@ Chevron Services Company, a division of Chevron U.S.A Inc. | Houston, TX

Faculty members in Research areas such as Bayesian and Spatial Statistics; Data Privacy and Security; AI/ML; NLP; Image and Video Data Analysis

@ Ahmedabad University | Ahmedabad, India

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL