Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain. (arXiv:2108.03001v2 [cs.CV] UPDATED)

One of the key challenges in Neural Architecture Search (NAS) is to
efficiently rank the performances of architectures. The mainstream assessment
of performance rankers uses ranking correlations (e.g., Kendall's tau), which
pay equal attention to the whole space. However, the optimization goal of NAS
is identifying top architectures while paying less attention on other
architectures in the search space. In this paper, we show both empirically and
theoretically that Normalized Discounted Cumulative Gain (NDCG) is a better
metric for rankers. …

arxiv neural architectures