Instance-Level Trojan Attacks on Visual Question Answering via Adversarial Learning in Neuron Activation Space

arXiv:2304.00436v2 Announce Type: replace
Abstract: Trojan attacks embed perturbations in input data leading to malicious behavior in neural network models. A combination of various Trojans in different modalities enables an adversary to mount a sophisticated attack on multimodal learning such as Visual Question Answering (VQA). However, multimodal Trojans in conventional methods are susceptible to parameter adjustment during processes such as fine-tuning. To this end, we propose an instance-level multimodal Trojan attack on VQA that efficiently adapts to fine-tuned models through …

abstract adversarial adversarial learning arxiv attacks behavior combination cs.ai cs.cv data embed however instance multimodal multimodal learning network neural network neuron question question answering space type via visual vqa