Web: http://arxiv.org/abs/2206.08558

June 20, 2022, 1:10 a.m. | Sameera Ramasinghe, Lachlan MacDonald, Moshiur Farazi, Hemanth Sartachandran, Simon Lucey

cs.LG updates on arXiv.org arxiv.org

Characterizing the remarkable generalization properties of over-parameterized
neural networks remains an open problem. In this paper, we promote a shift of
focus towards initialization rather than neural architecture or (stochastic)
gradient descent to explain this implicit regularization. Through a Fourier
lens, we derive a general result for the spectral bias of neural networks and
show that the generalization of neural networks is heavily tied to their
initialization. Further, we empirically solidify the developed theoretical
insights using practical, deep networks. Finally, …

arxiv lg

More from arxiv.org / cs.LG updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY