Level Set Teleportation: An Optimization Perspective

arXiv:2403.03362v1 Announce Type: new
Abstract: We study level set teleportation, an optimization sub-routine which seeks to accelerate gradient methods by maximizing the gradient norm on a level-set of the objective function. Since the descent lemma implies that gradient descent (GD) decreases the objective proportional to the squared norm of the gradient, level-set teleportation maximizes this one-step progress guarantee. For convex functions satisfying Hessian stability, we prove that GD with level-set teleportation obtains a combined sub-linear/linear convergence rate which is strictly …

abstract arxiv cs.lg function gradient math.oc norm optimization perspective set study teleportation type