Web: http://arxiv.org/abs/2208.04924

Aug. 12, 2022, 1:11 a.m. | Qingguo Hong, Qinyang Tan, Jonathan W. Siegel, Jinchao Xu

cs.LG updates on arXiv.org arxiv.org

Neural networks are universal function approximators which are known to
generalize well despite being dramatically overparameterized. We study this
phenomenon from the point of view of the spectral bias of neural networks. Our
contributions are two-fold. First, we provide a theoretical explanation for the
spectral bias of ReLU neural networks by leveraging connections with the theory
of finite element methods. Second, based upon this theory we predict that
switching the activation function to a piecewise linear B-spline, namely the
Hat …

arxiv bias function lg networks neural networks

More from arxiv.org / cs.LG updates on arXiv.org

Research Scientists

@ ODU Research Foundation | Norfolk, Virginia

Embedded Systems Engineer (Robotics)

@ Neo Cybernetica | Bedford, New Hampshire

2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

@ Lawrence Berkeley National Lab | San Francisco, CA

Senior Manager Data Scientist

@ NAV | Remote, US

Senior AI Research Scientist

@ Earth Species Project | Remote anywhere

Research Fellow- Center for Security and Emerging Technology (Multiple Opportunities)

@ University of California Davis | Washington, DC

Staff Fellow - Data Scientist

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Staff Fellow - Senior Data Engineer

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Machine Learning Data Engineer Intern (Jyoti Dharna)

@ Benson Hill | St. Louis, Missouri

Software Engineer / SDE I, Chime SDK Video Research Engineering

@ Amazon.com | East Palo Alto, California, USA

IND (New) Senior ML Ops Engineer - WiQ

@ Quantium | Hyderabad

Data Engineer

@ LendingTree | Remote