TG-NAS: Leveraging Zero-Cost Proxies with Transformer and Graph Convolution Networks for Efficient Neural Architecture Search

arXiv:2404.00271v1 Announce Type: new
Abstract: Neural architecture search (NAS) is an effective method for discovering new convolutional neural network (CNN) architectures. However, existing approaches often require time-consuming training or intensive sampling and evaluations. Zero-shot NAS aims to create training-free proxies for architecture performance prediction. However, existing proxies have suboptimal performance, and are often outperformed by simple metrics such as model parameter counts or the number of floating-point operations. Besides, existing model-based proxies cannot be generalized to new search spaces with …

abstract architecture architectures arxiv cnn convolution convolutional neural network cost cs.ai cs.lg free graph however nas network networks neural architecture search neural network performance prediction proxies sampling search training transformer type zero-shot