An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning

arXiv:2308.10098v2 Announce Type: replace-cross
Abstract: Various tasks in data science are modeled utilizing the variational regularization approach, where manually selecting regularization parameters presents a challenge. The difficulty gets exacerbated when employing regularizers involving a large number of hyperparameters. To overcome this challenge, bilevel learning can be employed to learn such parameters from data. However, neither exact function values nor exact gradients with respect to the hyperparameters are attainable, necessitating methods that only rely on inexact evaluation of such quantities. State-of-the-art …

abstract application arxiv challenge cs.lg data data science hyperparameter math.oc optimization parameters regularization science tasks type