From Optimization Dynamics to Generalization Bounds via {\L}ojasiewicz Gradient Inequality. (arXiv:2202.10670v3 [stat.ML] UPDATED)

Optimization and generalization are two essential aspects of statistical
machine learning. In this paper, we propose a framework to connect optimization
with generalization by analyzing the generalization error based on the
optimization trajectory under the gradient flow algorithm. The key ingredient
of this framework is the Uniform-LGI, a property that is generally satisfied
when training machine learning models. Leveraging the Uniform-LGI, we first
derive convergence rates for gradient flow algorithm, then we give
generalization bounds for a large class of …

arxiv dynamics gradient inequality optimization