MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models

arXiv:2403.19080v1 Announce Type: new
Abstract: Different from a unimodal model whose input is from a single modality, the input (called multi-modal input) of a multi-modal model is from multiple modalities such as image, 3D points, audio, text, etc. Similar to unimodal models, many existing studies show that a multi-modal model is also vulnerable to adversarial perturbation, where an attacker could add small perturbation to all modalities of a multi-modal input such that the multi-modal model makes incorrect predictions for it. …

abstract adversarial adversarial attacks arxiv attacks audio cs.cr cs.cv defense etc image modal multi-modal multiple show studies text type