BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning

arXiv:2311.12075v3 Announce Type: replace
Abstract: Studying backdoor attacks is valuable for model copyright protection and enhancing defenses. While existing backdoor attacks have successfully infected multimodal contrastive learning models such as CLIP, they can be easily countered by specialized backdoor defenses for MCL models. This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses and introduces the \emph{\toolns} attack, which is resistant to backdoor detection and model fine-tuning defenses. To achieve this, we …

abstract arxiv attacks backdoor clip copyright copyright protection cs.cv embedding multimodal paper practical protection studying threats type