Teach LLMs to Phish: Stealing Private Information from Language Models

arXiv:2403.00871v1 Announce Type: cross
Abstract: When large language models are trained on private data, it can be a significant privacy risk for them to memorize and regurgitate sensitive information. In this work, we propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data with upwards of 10% attack success rates, at times, …

abstract arxiv call cs.ai cs.cl cs.cr cs.lg data data extraction extraction information language language models large language large language models llms phishing practical privacy private data risk stealing them type work