Encodings for Prediction-based Neural Architecture Search

arXiv:2403.02484v1 Announce Type: new
Abstract: Predictor-based methods have substantially enhanced Neural Architecture Search (NAS) optimization. The efficacy of these predictors is largely influenced by the method of encoding neural network architectures. While traditional encodings used an adjacency matrix describing the graph structure of a neural network, novel encodings embrace a variety of approaches from unsupervised pretraining of latent representations to vectors of zero-cost proxies. In this paper, we categorize and investigate neural encodings from three main types: structural, learned, and …

architecture arxiv cs.ai cs.cv cs.lg cs.ne neural architecture search prediction search type