Towards Extremely Fast Bilevel Optimization with Self-governed Convergence Guarantees. (arXiv:2205.10054v1 [math.OC])

Gradient methods have become mainstream techniques for Bi-Level Optimization
(BLO) in learning and vision fields. The validity of existing works heavily
relies on solving a series of approximation subproblems with extraordinarily
high accuracy. Unfortunately, to achieve the approximation accuracy requires
executing a large quantity of time-consuming iterations and computational
burden is naturally caused. This paper is thus devoted to address this critical
computational issue. In particular, we propose a single-level formulation to
uniformly understand existing explicit and implicit Gradient-based BLOs …

arxiv convergence math optimization