Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

We propose the Data Contamination Quiz (DCQ), a simple and effective approach to detect data contamination in large language models (LLMs) and estimate the amount of it. Specifically, we frame data contamination detection as a series of multiple-choice questions and devise a quiz format wherein three perturbed versions of each dataset instance are created. These changes only include word-level perturbations. The generated perturbed versions, along with the original instance, form the options in the DCQ, with an extra option accommodating …

cs.ai cs.cl cs.lg data detection format language language models large language large language models llms multiple questions quiz series simple tool