all AI news
Gradient-based Bi-level Optimization for Deep Learning: A Survey. (arXiv:2207.11719v2 [cs.LG] UPDATED)
cs.LG updates on arXiv.org arxiv.org
Bi-level optimization, especially the gradient-based category, has been
widely used in the deep learning community including hyperparameter
optimization and meta knowledge extraction. Bi-level optimization embeds one
problem within another and the gradient-based category solves the outer level
task by computing the hypergradient, which is much more efficient than
classical methods such as the evolutionary algorithm. In this survey, we first
give a formal definition of the gradient-based bi-level optimization. Secondly,
we illustrate how to formulate a research problem as a …
arxiv bi deep learning gradient learning lg optimization survey