Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory

arXiv:2403.12445v1 Announce Type: new
Abstract: Vision-language pre-training (VLP) models exhibit remarkable capabilities in comprehending both images and text, yet they remain susceptible to multimodal adversarial examples (AEs). Strengthening adversarial attacks and uncovering vulnerabilities, especially common issues in VLP models (e.g., high transferable AEs), can stimulate further research on constructing reliable and practical VLP models. A recent work (i.e., Set-level guidance attack) indicates that augmenting image-text pairs to increase AE diversity along the optimization path enhances the transferability of adversarial examples …

abstract adversarial adversarial attacks adversarial examples arxiv attacks boosting capabilities cs.cv diversification examples images intersection language multimodal pre-training text training trajectory type via vision vulnerabilities