Tired of Tuning Learning Rates? Meet DoG: A Simple Parameter-Free Optimizer Backed by Solid Theoretical Guarantees

Researchers at Tel Aviv University propose tuning free dynamic SGD step size formula, called Distance over Gradients (DoG), which only depends upon the empirical quantities with no learning rate parameter. They theoretically show that a slight variation in the DoG formula would lead to locally bounded stochastic gradients converging.  A stochastic process requires an optimized […]

The post Tired of Tuning Learning Rates? Meet DoG: A Simple Parameter-Free Optimizer Backed by Solid Theoretical Guarantees appeared first on MarkTechPost.

ai shorts applications artificial intelligence dynamic editors pick free machine learning rate researchers show simple solid staff technology tel aviv tel aviv university university variation