Sharpened Lazy Incremental Quasi-Newton Method

arXiv:2305.17283v2 Announce Type: replace-cross
Abstract: The problem of minimizing the sum of $n$ functions in $d$ dimensions is ubiquitous in machine learning and statistics. In many applications where the number of observations $n$ is large, it is necessary to use incremental or stochastic methods, as their per-iteration cost is independent of $n$. Of these, Quasi-Newton (QN) methods strike a balance between the per-iteration cost and the convergence rate. Specifically, they exhibit a superlinear rate with $O(d^2)$ cost in contrast to …

abstract applications arxiv cost cs.lg dimensions functions incremental independent iteration lazy machine machine learning math.oc per statistics stochastic type