PETA: Parameter-Efficient Trojan Attacks

arXiv:2310.00648v4 Announce Type: replace
Abstract: Parameter-efficient fine-tuning (PEFT) enables efficient adaptation of pre-trained language models (PLMs) to specific tasks. By tuning only a minimal set of (extra) parameters, PEFT achieves performance that is comparable to standard fine-tuning. However, despite its prevalent use, the security implications of PEFT remain largely unexplored. In this paper, we take the initial steps and present PETA, a novel trojan attack that compromises the weights of PLMs by accounting for downstream adaptation through bilevel optimization: the …

abstract arxiv attacks cs.cl extra fine-tuning language language models paper parameters peft performance security set specific tasks standard tasks type