Perturbation Analysis of Neural Collapse. (arXiv:2210.16658v1 [cs.LG])

Training deep neural networks for classification often includes minimizing
the training loss beyond the zero training error point. In this phase of
training, a "neural collapse" behavior has been observed: the variability of
features (outputs of the penultimate layer) of within-class samples decreases
and the mean features of different classes approach a certain tight frame
structure. Recent works analyze this behavior via idealized unconstrained
features models where all the minimizers exhibit exact collapse. However, with
practical networks and datasets, the …

analysis arxiv neural collapse