User Inference Attacks on Large Language Models

arXiv:2310.09266v2 Announce Type: replace-cross
Abstract: Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and …

abstract applications arxiv attacks cs.cl cs.cr cs.lg data fine-tuning inference language language models large language large language models llms paper privacy study tasks threat type user data