Web: http://arxiv.org/abs/2206.08684

June 20, 2022, 1:13 a.m. | Zheng He, Zeke Xie, Quanzhi Zhu, Zengchang Qin

cs.CV updates on arXiv.org arxiv.org

People usually believe that network pruning not only reduces the
computational cost of deep networks, but also prevents overfitting by
decreasing model capacity. However, our work surprisingly discovers that
network pruning sometimes even aggravates overfitting. We report an unexpected
sparse double descent phenomenon that, as we increase model sparsity via
network pruning, test performance first gets worse (due to overfitting), then
gets better (due to relieved overfitting), and gets worse at last (due to
forgetting useful information). While recent studies …

arxiv lg network overfitting pruning

More from arxiv.org / cs.CV updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY