Talk Too Much: Poisoning Large Language Models under Token Limit

arXiv:2404.14795v1 Announce Type: cross
Abstract: Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by …

abstract arxiv attacks cs.cl cs.cr cs.lg detection however human instance language language models large language large language models llms poisoning attacks queries responses set talk token type words world