Kernel Normalized Convolutional Networks. (arXiv:2205.10089v1 [cs.LG])

Existing deep convolutional neural network (CNN) architectures frequently
rely upon batch normalization (BatchNorm) to effectively train the model.
BatchNorm significantly improves model performance, but performs poorly with
smaller batch sizes. To address this limitation, we propose kernel
normalization and kernel normalized convolutional layers, and incorporate them
into kernel normalized convolutional networks (KNConvNets) as the main building
blocks. We implement KNConvNets corresponding to the state-of-the-art CNNs such
as ResNet and DenseNet while forgoing BatchNorm layers. Through extensive
experiments, we illustrate that …

arxiv kernel networks