BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP

arXiv:2311.16194v2 Announce Type: replace
Abstract: Contrastive Vision-Language Pre-training, known as CLIP, has shown promising effectiveness in addressing downstream image recognition tasks. However, recent works revealed that the CLIP model can be implanted with a downstream-oriented backdoor. On downstream tasks, one victim model performs well on clean samples but predicts a specific target class whenever a specific trigger is present. For injecting a backdoor, existing attacks depend on a large amount of additional data to maliciously fine-tune the entire pre-trained CLIP …

abstract arxiv attacks backdoor clip cs.cv however image image recognition implanted language pre-training prompt prompt learning recognition samples tasks training type vision