First-order penalty methods for bilevel optimization

arXiv:2301.01716v2 Announce Type: replace-cross
Abstract: In this paper we study a class of unconstrained and constrained bilevel optimization problems in which the lower level is a possibly nonsmooth convex optimization problem, while the upper level is a possibly nonconvex optimization problem. We introduce a notion of $\varepsilon$-KKT solution for them and show that an $\varepsilon$-KKT solution leads to an $O(\sqrt{\varepsilon})$- or $O(\varepsilon)$-hypergradient based stionary point under suitable assumptions. We also propose first-order penalty methods for finding an $\varepsilon$-KKT solution of …

abstract arxiv class cs.lg cs.na kkt math.na math.oc notion optimization paper show solution stat.ml study them type