all AI news
Randomized K-FACs: Speeding up K-FAC with Randomized Numerical Linear Algebra. (arXiv:2206.15397v1 [cs.LG])
July 1, 2022, 1:11 a.m. | Constantin Octavian Puiu
stat.ML updates on arXiv.org arxiv.org
K-FAC is a successful tractable implementation of Natural Gradient for Deep
Learning, which nevertheless suffers from the requirement to compute the
inverse of the Kronecker factors (through an eigen-decomposition). This can be
very time-consuming (or even prohibitive) when these factors are large. In this
paper, we theoretically show that, owing to the exponential-average
construction paradigm of the Kronecker factors that is typically used, their
eigen-spectrum must decay. We show numerically that in practice this decay is
very rapid, leading to …
More from arxiv.org / stat.ML updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Data Scientist
@ Motive | India - Remote
Senior Perception Engineer
@ NVIDIA | US, CA, Santa Clara
Business Data Analyst, Finance and Treasury Data Repositories, Senior Associate
@ State Street | Krakow, Poland
Junior AI Engineer (Internship)
@ Sony | SEU - Italy - Roma
Manager, Data Science 3
@ PayPal | USA - Pennsylvania - Virtual