Investigating the Impact of Model Width and Density on Generalization in Presence of Label Noise

arXiv:2208.08003v5 Announce Type: replace
Abstract: Increasing the size of overparameterized neural networks has been a key in achieving state-of-the-art performance. This is captured by the double descent phenomenon, where the test loss follows a decreasing-increasing-decreasing pattern (or sometimes monotonically decreasing) as model width increases. However, the effect of label noise on the test loss curve has not been fully explored. In this work, we uncover an intriguing phenomenon where label noise leads to a \textit{final ascent} in the originally observed …

abstract art arxiv cs.lg however impact key loss networks neural networks noise pattern performance state stat.ml test type