all AI news
Neural Tangent Kernel Analysis of Deep Narrow Neural Networks. (arXiv:2202.02981v2 [cs.LG] UPDATED)
June 29, 2022, 1:11 a.m. | Jongmin Lee, Joo Young Choi, Ernest K. Ryu, Albert No
stat.ML updates on arXiv.org arxiv.org
The tremendous recent progress in analyzing the training dynamics of
overparameterized neural networks has primarily focused on wide networks and
therefore does not sufficiently address the role of depth in deep learning. In
this work, we present the first trainability guarantee of infinitely deep but
narrow neural networks. We study the infinite-depth limit of a multilayer
perceptron (MLP) with a specific initialization and establish a trainability
guarantee using the NTK theory. We then extend the analysis to an infinitely
deep …
More from arxiv.org / stat.ML updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Research Assistant/Associate, Health Data Science [LKCMedicine]
@ Nanyang Technological University | NTU Novena Campus, Singapore
Senior Machine Learning Engineer, Portfolio ML
@ Affirm | Remote Canada
[Sessional Lecturer] Foundations of Data Analytics and Machine Learning - APS1070
@ University of Toronto | Toronto, ON, CA
Senior Data Scientist
@ Prosper | United States
Data Analyst
@ ZF Friedrichshafen AG | Coimbatore, TN, IN, 641659