Training neural networks using monotone variational inequality. (arXiv:2202.08876v1 [stat.ML])

Despite the vast empirical success of neural networks, theoretical
understanding of the training procedures remains limited, especially in
providing performance guarantees of testing performance due to the non-convex
nature of the optimization problem. Inspired by a recent work of (Juditsky &
Nemirovsky, 2019), instead of using the traditional loss function minimization
approach, we reduce the training of the network parameters to another problem
with convex structure -- to solve a monotone variational inequality (MVI). The
solution to MVI can be …

arxiv inequality ml networks neural networks training