On the High Symmetry of Neural Network Functions. (arXiv:2211.06603v1 [cs.LG])

Training neural networks means solving a high-dimensional optimization
problem. Normally the goal is to minimize a loss function that depends on what
is called the network function, or in other words the function that gives the
network output given a certain input. This function depends on a large number
of parameters, also known as weights, that depends on the network architecture.
In general the goal of this optimization problem is to find the global minimum
of the network function. In …

arxiv network neural network symmetry