Can We Understand Plasticity Through Neural Collapse?

arXiv:2404.02719v1 Announce Type: new
Abstract: This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting.

abstract analyze arxiv association correlation cs.ai cs.lg deep learning loss neural collapse paper regularization through training type