An alternative approach to train neural networks using monotone variational inequality. (arXiv:2202.08876v3 [stat.ML] UPDATED)

Despite the vast empirical success of neural networks, theoretical
understanding of the training procedures remains limited, especially in
providing performance guarantees of testing performance due to the non-convex
nature of the optimization problem. The current paper investigates an
alternative approach of neural network training by reducing to another problem
with convex structure -- to solve a monotone variational inequality (MVI) --
inspired by a recent work of (Juditsky & Nemirovsky, 2019). The solution to MVI
can be found by computationally …

arxiv inequality ml networks neural networks