Cutting through buggy adversarial example defenses: fixing 1 line of code breaks Sabre

arXiv:2405.03672v1 Announce Type: cross
Abstract: Sabre is a defense to adversarial examples that was accepted at IEEE S&P 2024. We first reveal significant flaws in the evaluation that point to clear signs of gradient masking. We then show the cause of this gradient masking: a bug in the original evaluation code. By fixing a single line of code in the original repository, we reduce Sabre's robust accuracy to 0%. In response to this, the authors modify the defense and introduce …

abstract adversarial adversarial examples arxiv clear code cs.cr cs.lg defense evaluation example examples flaws gradient ieee line masking show through type