Why is SAM Robust to Label Noise?

arXiv:2405.03676v1 Announce Type: new
Abstract: Sharpness-Aware Minimization (SAM) is most known for achieving state-of the-art performances on natural image and language tasks. However, its most pronounced improvements (of tens of percent) is rather in the presence of label noise. Understanding SAM's label noise robustness requires a departure from characterizing the robustness of minimas lying in "flatter" regions of the loss landscape. In particular, the peak performance under label noise occurs with early stopping, far before the loss converges. We decompose …

abstract art arxiv cs.lg however image improvements language natural noise performances robust robustness sam state tasks type understanding