A Common Misconception About Cross Entropy Loss

**Cross Entropy Loss for multi class classification, when the last layer is Softmax**

The misconception is that the network *only* learns from its prediction on the correct class

It is common online to see comments [like this one](https://datascience.stackexchange.com/a/20301/159699), that, while technically true, obfuscate the understanding of how a neural network updates its parameters after training on a single sample in multi-class classification. Other comments, [such as this one](https://datascience.stackexchange.com/a/31966/159699), [and this one](https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation/24696#comment91209_24696), are flat out wrong. This makes studying this topic …

class classification datascience entropy function layer loss network prediction sample softmax