Feb. 11, 2024, 5:38 p.m. | /u/Toasty_toaster

Data Science www.reddit.com

**Cross Entropy Loss for multi class classification, when the last layer is Softmax**

The misconception is that the network *only* learns from its prediction on the correct class

It is common online to see comments [like this one](https://datascience.stackexchange.com/a/20301/159699), that, while technically true, obfuscate the understanding of how a neural network updates its parameters after training on a single sample in multi-class classification. Other comments, [such as this one](https://datascience.stackexchange.com/a/31966/159699), [and this one](https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation/24696#comment91209_24696), are flat out wrong. This makes studying this topic …

class classification datascience entropy function layer loss network prediction sample softmax

Research Scholar (Technical Research)

@ Centre for the Governance of AI | Hybrid; Oxford, UK

HPC Engineer (x/f/m) - DACH

@ Meshcapade GmbH | Remote, Germany

Encounter Data Management Professional

@ Humana | Work at Home - Kentucky

Pre-sales Manager (Data, Analytics & AI)

@ Databricks | Stockholm, Sweden

Lecturer / Senior Lecturer - Medical Imaging

@ Central Queensland University | Mackay, QLD, AU

Intern - Research Engineer

@ Plus | Santa Clara, CA