Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning

arXiv:2402.12168v1 Announce Type: cross
Abstract: Recently, various parameter-efficient fine-tuning (PEFT) strategies for application to language models have been proposed and successfully implemented. However, this raises the question of whether PEFT, which only updates a limited set of model parameters, constitutes security vulnerabilities when confronted with weight-poisoning backdoor attacks. In this study, we show that PEFT is more susceptible to weight-poisoning backdoor attacks compared to the full-parameter fine-tuning method, with pre-defined triggers remaining exploitable and pre-defined targets maintaining high confidence, even …

abstract application arxiv attacks backdoor cs.ai cs.cl cs.cr fine-tuning language language models parameters peft question raises security security vulnerabilities set strategies study type updates vulnerabilities