AdvLoRA: Adversarial Low-Rank Adaptation of Vision-Language Models

arXiv:2404.13425v1 Announce Type: new
Abstract: Vision-Language Models (VLMs) are a significant technique for Artificial General Intelligence (AGI). With the fast growth of AGI, the security problem become one of the most important challenges for VLMs. In this paper, through extensive experiments, we demonstrate the vulnerability of the conventional adaptation methods for VLMs, which may bring significant security risks. In addition, as the size of the VLMs increases, performing conventional adversarial adaptation techniques on VLMs results in high computational costs. To …

abstract adversarial agi artificial artificial general intelligence arxiv become challenges cs.ai cs.cv general growth intelligence language language models low low-rank adaptation paper security through type vision vision-language vision-language models vlms vulnerability