Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images

arXiv:2401.11170v2 Announce Type: replace
Abstract: Large vision-language models (VLMs) such as GPT-4 have achieved exceptional performance across various multi-modal tasks. However, the deployment of VLMs necessitates substantial energy consumption and computational resources. Once attackers maliciously induce high energy consumption and latency time (energy-latency cost) during inference of VLMs, it will exhaust computational resources. In this paper, we explore this attack surface about availability of VLMs and aim to induce high energy-latency cost during inference of VLMs. We find that high …

arxiv cs.cr cs.cv energy images language language models latency type vision vision-language models