Effective Prompt Extraction from Language Models

arXiv:2307.06865v2 Announce Type: replace
Abstract: The text generated by large language models is commonly controlled by prompting, where a prompt prepended to a user's query guides the model's output. The prompts used by companies to guide their models are often treated as secrets, to be hidden from the user making the query. They have even been treated as commodities to be bought and sold. However, anecdotal reports have shown adversarial users employing prompt extraction attacks to recover these prompts. In …

abstract arxiv companies cs.ai cs.cl extraction generated guide guides hidden language language models large language large language models making prompt prompting prompts query text type