Efficient implementation of incremental proximal-point methods. (arXiv:2205.01457v1 [cs.LG])

Web: http://arxiv.org/abs/2205.01457

Model training algorithms which observe a small portion of the training set
in each computational step are ubiquitous in practical machine learning, and
include both stochastic and online optimization methods. In the vast majority
of cases, such algorithms typically observe the training samples via the
gradients of the cost functions the samples incur. Thus, these methods exploit
are the \emph{slope} of the cost functions via their first-order
approximations.

To address limitations of gradient-based methods, such as sensitivity to
step-size choice …

arxiv implementation incremental