Learning to optimize with convergence guarantees using nonlinear system theory

arXiv:2403.09389v1 Announce Type: cross
Abstract: The increasing reliance on numerical methods for controlling dynamical systems and training machine learning models underscores the need to devise algorithms that dependably and efficiently navigate complex optimization landscapes. Classical gradient descent methods offer strong theoretical guarantees for convex problems; however, they demand meticulous hyperparameter tuning for non-convex ones. The emerging paradigm of learning to optimize (L2O) automates the discovery of algorithms with optimized performance leveraging learning models and data - yet, it lacks a …

abstract algorithms arxiv convergence cs.lg cs.sy demand eess.sy gradient however hyperparameter machine machine learning machine learning models numerical optimization reliance systems theory training type