FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness

arXiv:2402.19102v1 Announce Type: new
Abstract: Neural Architecture Search (NAS) paves the way for the automatic definition of Neural Network (NN) architectures, attracting increasing research attention and offering solutions in various scenarios. This study introduces a novel NAS solution, called Flat Neural Architecture Search (FlatNAS), which explores the interplay between a novel figure of merit based on robustness to weight perturbations and single NN optimization with Sharpness-Aware Minimization (SAM). FlatNAS is the first work in the literature to systematically explore flat …

abstract architecture architectures arxiv attention cs.ai cs.cv cs.lg definition distribution nas network neural architecture search neural network novel research robustness search solution solutions study the way type