A New Study by Google and DeepMind Introduces Geometric Complexity (GC) for Neural Network Analysis and Understanding of Deep Learning Models

✔ Research team developed a computationally tractable notion of complexity, which they call Geometric Complexity (GC), that has many close relationships with many areas in deep learning and mathematics, including harmonic function theory, Lipschitz smoothness, and regularization theory.

✔ They provide evidence that common training heuristics keep the geometric complexity low, including (i) common initialization schemes (ii) the use of over parametrized models with a large number of layers (Fig. 2), (iii) large learning rates, small batch sizes, and implicit …

analysis complexity deep learning deepmind geometric complexity google machinelearningnews network neural network study understanding