Overview frequency principle/spectral bias in deep learning. (arXiv:2201.07395v1 [cs.LG])

Understanding deep learning is increasingly emergent as it penetrates more
and more into industry and science. In recent years, a research line from
Fourier analysis sheds lights into this magical "black box" by showing a
Frequency Principle (F-Principle or spectral bias) of the training behavior of
deep neural networks (DNNs) -- DNNs often fit functions from low to high
frequency during the training. The F-Principle is first demonstrated by
one-dimensional synthetic data followed by the verification in high-dimensional
real datasets. …

arxiv bias deep learning learning