Web: http://arxiv.org/abs/2206.15397

Sept. 29, 2022, 1:13 a.m. | Constantin Octavian Puiu

stat.ML updates on arXiv.org arxiv.org

K-FAC is a successful tractable implementation of Natural Gradient for Deep
Learning, which nevertheless suffers from the requirement to compute the
inverse of the Kronecker factors (through an eigen-decomposition). This can be
very time-consuming (or even prohibitive) when these factors are large. In this
paper, we theoretically show that, owing to the exponential-average
construction paradigm of the Kronecker factors that is typically used, their
eigen-spectrum must decay. We show numerically that in practice this decay is
very rapid, leading to …

arxiv linear linear algebra numerical

More from arxiv.org / stat.ML updates on arXiv.org

Staff Data Scientist - Merchant Services (Remote, North America)

@ Shopify | Dallas, TX, United States

Machine Learning / Data Engineer

@ WATI | Vietnam - Remote

F/H Data Manager

@ Bosch Group | Saint-Ouen-sur-Seine, France

[Fixed-term contract until July 2023] Data Quality Controller - Space Industry Luxembourg (m/f/o)

@ LuxSpace Sarl | Betzdorf, Luxembourg

Senior Data Engineer (Azure DataBricks/datalake)

@ SpectraMedix | East Windsor, NJ, United States

Abschlussarbeit im Bereich Data Analytics (w/m/div.)

@ Bosch Group | Rülzheim, Germany

Data Engineer - Marketing

@ Publicis Groupe | London, United Kingdom

Data Engineer (Consulting division)

@ Starschema | Budapest, Hungary

Team Leader, Master Data Management - Support CN, HK & TW

@ Publicis Groupe | Kuala Lumpur, Malaysia

Senior Software Engineer (Big Data & Platform Team) - Data & AI

@ Allegro | Warszawa, Toruń, Kraków, Poznań, Poland

Développeur Big Data (H/F)

@ CITECH | Paris, France

Big Data Engineer - Data & AI

@ Allegro | Poznań, Warszawa, Kraków, Toruń, Wrocław, Gdańsk, Łódź, Poland