Improved Binary Forward Exploration: Learning Rate Scheduling Method for Stochastic Optimization. (arXiv:2207.04198v2 [cs.LG] UPDATED)

A new gradient-based optimization approach by automatically scheduling the
learning rate has been proposed recently, which is called Binary Forward
Exploration (BFE). The Adaptive version of BFE has also been discussed
thereafter. In this paper, the improved algorithms based on them will be
investigated, in order to optimize the efficiency and robustness of the new
methodology. This improved approach provides a new perspective to scheduling
the update of learning rate and will be compared with the stochastic gradient
descent, aka …

arxiv binary exploration learning lg optimization rate scheduling stochastic