Distilling Adversarial Robustness Using Heterogeneous Teachers

arXiv:2402.15586v1 Announce Type: new
Abstract: Achieving resiliency against adversarial attacks is necessary prior to deploying neural network classifiers in domains where misclassification incurs substantial costs, e.g., self-driving cars or medical imaging. Recent work has demonstrated that robustness can be transferred from an adversarially trained teacher to a student model using knowledge distillation. However, current methods perform distillation using a single adversarial and vanilla teacher and consider homogeneous architectures (i.e., residual networks) that are susceptible to misclassify examples from similar adversarial …

abstract adversarial adversarial attacks arxiv attacks cars classifiers costs cs.cr cs.cv current distillation domains driving imaging knowledge medical medical imaging network neural network prior resiliency robustness self-driving teachers type work