$L_2$BN: Enhancing Batch Normalization by Equalizing the $L_2$ Norms of Features. (arXiv:2207.02625v3 [cs.CV] UPDATED)

In this paper, we show that the difference in $l_2$ norms of sample features
can hinder batch normalization from obtaining more distinguished inter-class
features and more compact intra-class features. To address this issue, we
propose an intuitive but effective method to equalize the $l_2$ norms of sample
features. Concretely, we $l_2$-normalize each sample feature before batch
normalization, and therefore the features are of the same magnitude. Since the
proposed method combines the $l_2$ normalization and batch normalization, we
name our …

arxiv features normalization