all AI news
The Multiscale Structure of Neural Network Loss Functions: The Effect on Optimization and Origin. (arXiv:2204.11326v2 [cs.LG] UPDATED)
May 12, 2022, 1:11 a.m. | Chao Ma, Daniel Kunin, Lei Wu, Lexing Ying
cs.LG updates on arXiv.org arxiv.org
Local quadratic approximation has been extensively used to study the
optimization of neural network loss functions around the minimum. Though, it
usually holds in a very small neighborhood of the minimum, and cannot explain
many phenomena observed during the optimization process. In this work, we study
the structure of neural network loss functions and its implication on
optimization in a region beyond the reach of good quadratic approximation.
Numerically, we observe that neural network loss functions possesses a
multiscale structure, …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Data Architect
@ Western Digital | San Jose, CA, United States
Senior Data Scientist GenAI (m/w/d)
@ Deutsche Telekom | Bonn, Deutschland
Senior Data Engineer, Telco (Remote)
@ Lightci | Toronto, Ontario
Consultant Data Architect/Engineer H/F - Innovative Tech
@ Devoteam | Lyon, France
(Senior) ML Engineer / Software Engineer Machine Learning & AI (m/f/x) onsite or remote (in Germany or Austria)
@ Scalable GmbH | Wien, Germany