Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity

arXiv:2404.06391v1 Announce Type: new
Abstract: One of the most intriguing findings in the structure of neural network landscape is the phenomenon of mode connectivity: For two typical global minima, there exists a path connecting them without barrier. This concept of mode connectivity has played a crucial role in understanding important phenomena in deep learning.
In this paper, we conduct a fine-grained analysis of this connectivity phenomenon. First, we demonstrate that in the overparameterized case, the connecting path can be as …

abstract arxiv concept connectivity cs.lg global landscape network neural network path role star stat.ml them type understanding