AdamL: A fast adaptive gradient method incorporating loss function

**Paper**: [https://arxiv.org/abs/2312.15295](https://arxiv.org/abs/2312.15295)

**Abstract**:

>Adaptive first-order optimizers are fundamental tools in deep learning, although they may suffer from poor generalization due to the nonuniform gradient scaling. In this work, we propose **AdamL**, a novel variant of the Adam optimizer, that takes into account the loss function information to attain better generalization results. We provide sufficient conditions that together with the Polyak-Lojasiewicz inequality, ensure the linear convergence of AdamL. As a byproduct of our analysis, we prove similar convergence properties for the …

abstract adam computervision deep learning function gradient inequality information loss novel scaling together tools work