Few-Shot Adversarial Prompt Learning on Vision-Language Models

arXiv:2403.14774v1 Announce Type: cross
Abstract: The vulnerability of deep neural networks to imperceptible adversarial perturbations has attracted widespread attention. Inspired by the success of vision-language foundation models, previous efforts achieved zero-shot adversarial robustness by aligning adversarial visual features with text supervision. However, in practice, they are still unsatisfactory due to several issues, including heavy adaptation cost, suboptimal text supervision, and uncontrolled natural generalization capacity. In this paper, to address these issues, we propose a few-shot adversarial prompt framework where adapting …

abstract adversarial arxiv attention cs.cl cs.cr cs.cv cs.lg features few-shot foundation however language language models networks neural networks practice prompt prompt learning robustness success supervision text type vision vision-language models visual vulnerability zero-shot