A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning

arXiv:2309.04877v2 Announce Type: replace
Abstract: The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift in focus from pattern recognition to decision-making and multi-agent problems. In these broader settings, new mathematical challenges emerge that involve equilibria and game theory instead of optima. Gradient-based methods remain essential -- given the high dimensionality and large scale of machine-learning problems -- but simple gradient descent …

abstract agent arxiv cs.lg decision focus gradient introduction machine machine learning making multi-agent optimization part pattern recognition productive progress recognition shift stat.ml type