A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

arXiv:2305.20028v2 Announce Type: replace
Abstract: Bayesian optimization is a highly efficient approach to optimizing objective functions which are expensive to query. These objectives are typically represented by Gaussian process (GP) surrogate models which are easy to optimize and support exact inference. While standard GP surrogates have been well-established in Bayesian optimization, Bayesian neural networks (BNNs) have recently become practical function approximators, with many benefits over standard GPs such as the ability to naturally handle non-stationarity and learn representations for high-dimensional …

abstract arxiv bayesian cs.lg easy functions inference network neural network optimization process query standard stat.ml study support type while