Speeding up Multi-objective Non-hierarchical Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-structured Parzen Estimator. (arXiv:2212.06751v2 [cs.LG] UPDATED)

Hyperparameter optimization (HPO) is a vital step in improving performance in
deep learning (DL). Practitioners are often faced with the trade-off between
multiple criteria, such as accuracy and latency. Given the high computational
needs of DL and the growing demand for efficient HPO, the acceleration of
multi-objective (MO) optimization becomes ever more important. Despite the
significant body of work on meta-learning for HPO, existing methods are
inapplicable to MO tree-structured Parzen estimator (MO-TPE), a simple yet
powerful MO-HPO algorithm. In …

accuracy algorithm arxiv computational deep learning demand hierarchical hyperparameter latency meta meta-learning multiple optimization paper performance trade tree work