The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks. (arXiv:2210.03820v1 [cs.LG])

In this work, we explore the maximum-margin bias of quasi-homogeneous neural
networks trained with gradient flow on an exponential loss and past a point of
separability. We introduce the class of quasi-homogeneous models, which is
expressive enough to describe nearly all neural networks with homogeneous
activations, even those with biases, residual connections, and normalization
layers, while structured enough to enable geometric analysis of its gradient
dynamics. Using this analysis, we generalize the existing results of
maximum-margin bias for homogeneous networks …

arxiv bias maximum margin networks neural networks