SCORE: Approximating Curvature Information under Self-Concordant Regularization. (arXiv:2112.07344v2 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2112.07344

In this paper, we propose the SCORE (self-concordant regularization)
framework for unconstrained minimization problems which incorporates
second-order information in the Newton decrement framework for convex
optimization. We propose the generalized Gauss-Newton with Self-Concordant
Regularization (GGN-SCORE) algorithm that updates the minimization variables
each time it receives a new input batch. The proposed algorithm exploits the
structure of the second-order information in the Hessian matrix, thereby
reducing computational overhead. GGN-SCORE demonstrates how we may speed up
convergence while also improving model generalization …

arxiv information lg regularization