Mitigating Neural Network Overconfidence with Logit Normalization. (arXiv:2205.09310v2 [cs.LG] UPDATED)

Detecting out-of-distribution inputs is critical for safe deployment of
machine learning models in the real world. However, neural networks are known
to suffer from the overconfidence issue, where they produce abnormally high
confidence for both in- and out-of-distribution inputs. In this work, we show
that this issue can be mitigated through Logit Normalization (LogitNorm) -- a
simple fix to the cross-entropy loss -- by enforcing a constant vector norm on
the logits in training. Our method is motivated by the …

arxiv lg network neural network normalization