Efficient NAS with FaDE on Hierarchical Spaces

arXiv:2404.16218v1 Announce Type: cross
Abstract: Neural architecture search (NAS) is a challenging problem. Hierarchical search spaces allow for cheap evaluations of neural network sub modules to serve as surrogate for architecture evaluations. Yet, sometimes the hierarchy is too restrictive or the surrogate fails to generalize. We present FaDE which uses differentiable architecture search to obtain relative performance predictions on finite regions of a hierarchical NAS space. The relative nature of these ranks calls for a memory-less, batch-wise outer search algorithm …

abstract architecture arxiv cs.ai cs.lg cs.ne differentiable hierarchical modules nas network neural architecture search neural network restrictive search serve spaces type