Multi-objective Differentiable Neural Architecture Search

arXiv:2402.18213v1 Announce Type: new
Abstract: Pareto front profiling in multi-objective optimization (MOO), i.e. finding a diverse set of Pareto optimal solutions, is challenging, especially with expensive objectives like neural network training. Typically, in MOO neural architecture search (NAS), we aim to balance performance and hardware metrics across devices. Prior NAS approaches simplify this task by incorporating hardware constraints into the objective function, but profiling the Pareto front necessitates a search for each constraint. In this work, we propose a novel …

abstract aim architecture arxiv balance cs.cv cs.lg devices differentiable diverse hardware metrics multi-objective nas network network training neural architecture search neural network optimization pareto performance prior profiling search set solutions stat.ml training type