Are All Losses Created Equal: A Neural Collapse Perspective. (arXiv:2210.02192v1 [cs.LG])

While cross entropy (CE) is the most commonly used loss to train deep neural
networks for classification tasks, many alternative losses have been developed
to obtain better empirical performance. Among them, which one is the best to
use is still a mystery, because there seem to be multiple factors affecting the
answer, such as properties of the dataset, the choice of network architecture,
and so on. This paper studies the choice of loss function by examining the
last-layer features of …

arxiv losses neural collapse perspective