StochGradAdam: Accelerating Neural Networks Training with Stochastic Gradient Sampling

In the rapidly advancing domain of deep learning optimization, this paper unveils the StochGradAdam optimizer, a novel adaptation of the well-regarded Adam algorithm. Central to StochGradAdam is its gradient sampling technique. This method not only ensures stable convergence but also leverages the advantages of selective gradient consideration, fostering robust training by potentially mitigating the effects of noisy or outlier data and enhancing the exploration of the loss landscape for more dependable convergence. In both image classification and segmentation tasks, StochGradAdam …

accelerating neural networks adam advantages algorithm convergence cs.ai cs.cv cs.lg cs.ne deep learning domain gradient networks neural networks novel optimization paper robust sampling stochastic training