Cluster-Based Normalization Layer for Neural Networks

arXiv:2403.16798v1 Announce Type: new
Abstract: Deep learning faces significant challenges during the training of neural networks, including internal covariate shift, label shift, vanishing/exploding gradients, overfitting, and computational complexity. While conventional normalization methods, such as Batch Normalization, aim to tackle some of these issues, they often depend on assumptions that constrain their adaptability. Mixture Normalization faces computational hurdles in its pursuit of handling multiple Gaussian distributions.
This paper introduces Cluster-Based Normalization (CB-Norm) in two variants - Supervised Cluster-Based Normalization (SCB-Norm) and …

abstract adaptability aim arxiv assumptions challenges cluster complexity computational cs.ai cs.lg deep learning layer networks neural networks normalization overfitting shift training type