Flatten the Curve: Efficiently Training Low-Curvature Neural Networks. (arXiv:2206.07144v1 [cs.LG])

Web: http://arxiv.org/abs/2206.07144

The highly non-linear nature of deep neural networks causes them to be
susceptible to adversarial examples and have unstable gradients which hinders
interpretability. However, existing methods to solve these issues, such as
adversarial training, are expensive and often sacrifice predictive accuracy.

In this work, we consider curvature, which is a mathematical quantity which
encodes the degree of non-linearity. Using this, we demonstrate low-curvature
neural networks (LCNNs) that obtain drastically lower curvature than standard
models while exhibiting similar predictive performance, which …

arxiv flatten flatten the curve lg networks neural neural networks training