Extended Unconstrained Features Model for Exploring Deep Neural Collapse. (arXiv:2202.08087v3 [cs.LG] UPDATED)

The modern strategy for training deep neural networks for classification
tasks includes optimizing the network's weights even after the training error
vanishes to further push the training loss toward zero. Recently, a phenomenon
termed "neural collapse" (NC) has been empirically observed in this training
procedure. Specifically, it has been shown that the learned features (the
output of the penultimate layer) of within-class samples converge to their
mean, and the means of different classes exhibit a certain tight frame
structure, which …

arxiv features neural collapse