Balanced softmax cross-entropy for incremental learning with and without memory. (arXiv:2103.12532v5 [cs.LG] UPDATED)

When incrementally trained on new classes, deep neural networks are subject
to catastrophic forgetting which leads to an extreme deterioration of their
performance on the old classes while learning the new ones. Using a small
memory containing few samples from past classes has shown to be an effective
method to mitigate catastrophic forgetting. However, due to the limited size of
the replay memory, there is a large imbalance between the number of samples for
the new and the old classes …

arxiv cross-entropy entropy incremental memory softmax