all AI news
Rigorous dynamical mean field theory for stochastic gradient descent methods. (arXiv:2210.06591v1 [math-ph])
Oct. 14, 2022, 1:11 a.m. | Cedric Gerbelot, Emanuele Troiani, Francesca Mignacco, Florent Krzakala, Lenka Zdeborova
cs.LG updates on arXiv.org arxiv.org
We prove closed-form equations for the exact high-dimensional asymptotics of
a family of first order gradient-based methods, learning an estimator (e.g.
M-estimator, shallow neural network, ...) from observations on Gaussian data
with empirical risk minimization. This includes widely used algorithms such as
stochastic gradient descent (SGD) or Nesterov acceleration. The obtained
equations match those resulting from the discretization of dynamical mean-field
theory (DMFT) equations from statistical physics when applied to gradient flow.
Our proof method allows us to give an …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior Data Engineer
@ Publicis Groupe | New York City, United States
Associate Principal Robotics Engineer - Research.
@ Dyson | United Kingdom - Hullavington Office
Duales Studium mit vertiefter Praxis: Bachelor of Science Künstliche Intelligenz und Data Science (m/w/d)
@ Gerresheimer | Wackersdorf, Germany
AI/ML Engineer (TS/SCI) {S}
@ ARKA Group, LP | Aurora, Colorado, United States
Data Integration Engineer
@ Find.co | Sliema
Data Engineer
@ Q2 | Bengaluru, India