Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defense

arXiv:2402.18787v1 Announce Type: new
Abstract: Recent studies have revealed the vulnerability of Deep Neural Networks (DNNs) to adversarial examples, which can easily fool DNNs into making incorrect predictions. To mitigate this deficiency, we propose a novel adversarial defense method called "Immunity" (Innovative MoE with MUtual information \& positioN stabilITY) based on a modified Mixture-of-Experts (MoE) architecture in this work. The key enhancements to the standard MoE are two-fold: 1) integrating of Random Switch Gates (RSGs) to obtain diverse network structures …

abstract adversarial adversarial examples arxiv cs.cr cs.lg defense examples experts information making moe networks neural networks novel predictions stability studies type vulnerability