PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor

arXiv:2403.06668v1 Announce Type: new
Abstract: Adversarial robustness of the neural network is a significant concern when it is applied to security-critical domains. In this situation, adversarial distillation is a promising option which aims to distill the robustness of the teacher network to improve the robustness of a small student network. Previous works pretrain the teacher network to make it robust to the adversarial examples aimed at itself. However, the adversarial examples are dependent on the parameters of the target network. …

abstract adversarial arxiv cs.cv cs.lg distillation domains network neural network peer robustness security small type