Proving membership in LLM pretraining data via data watermarks

arXiv:2402.10892v1 Announce Type: cross
Abstract: Detecting whether copyright holders' works were used in LLM pretraining is poised to be an important problem. This work proposes using data watermarks to enable principled detection with only black-box model access, provided that the rightholder contributed multiple training documents and watermarked them before public release. By applying a randomly sampled data watermark, detection can be framed as hypothesis testing, which provides guarantees on the false detection rate. We study two watermarks: one that inserts …

abstract arxiv box contributed copyright cs.cl cs.cr cs.lg data detection documents llm multiple pretraining public release them training type via watermarks work