all AI news
Black Box Lie Group Preconditioners for SGD. (arXiv:2211.04422v1 [stat.ML])
Nov. 9, 2022, 2:12 a.m. | Xilin Li
cs.LG updates on arXiv.org arxiv.org
A matrix free and a low rank approximation preconditioner are proposed to
accelerate the convergence of stochastic gradient descent (SGD) by exploiting
curvature information sampled from Hessian-vector products or finite
differences of parameters and gradients similar to the BFGS algorithm. Both
preconditioners are fitted with an online updating manner minimizing a
criterion that is free of line search and robust to stochastic gradient noise,
and further constrained to be on certain connected Lie groups to preserve their
corresponding symmetry or …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Senior Engineer - Data Science Operations
@ causaLens | London - Hybrid, England, United Kingdom
F0138 - LLM Developer (AI NLP)
@ Ubiquiti Inc. | Taipei
Staff Engineer, Database
@ Nagarro | Gurugram, India
Artificial Intelligence Assurance Analyst
@ Booz Allen Hamilton | USA, VA, McLean (8251 Greensboro Dr)