Web: http://arxiv.org/abs/2206.08257

June 17, 2022, 1:11 a.m. | Romain Cosson, Ali Jadbabaie, Anuran Makur, Amirhossein Reisizadeh, Devavrat Shah

cs.LG updates on arXiv.org arxiv.org

Several recent empirical studies demonstrate that important machine learning
tasks, e.g., training deep neural networks, exhibit low-rank structure, where
the loss function varies significantly in only a few directions of the input
space. In this paper, we leverage such low-rank structure to reduce the high
computational cost of canonical gradient-based methods such as gradient descent
(GD). Our proposed \emph{Low-Rank Gradient Descent} (LRGD) algorithm finds an
$\epsilon$-approximate stationary point of a $p$-dimensional function by first
identifying $r \leq p$ significant directions, …

arxiv gradient lg

More from arxiv.org / cs.LG updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY