Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs

arXiv:2403.04801v1 Announce Type: new
Abstract: In this paper, we introduce a black-box prompt optimization method that uses an attacker LLM agent to uncover higher levels of memorization in a victim agent, compared to what is revealed by prompting the target model with the training data directly, which is the dominant approach of quantifying memorization in LLMs. We use an iterative rejection-sampling optimization process to find instruction-based prompts with two main characteristics: (1) minimal overlap with the training data to avoid …

alpaca arxiv cs.cl llms type vicuna