Watermarking Makes Language Models Radioactive

arXiv:2402.14904v1 Announce Type: cross
Abstract: This paper investigates the radioactivity of LLM-generated texts, i.e. whether it is possible to detect that such input was used as training data. Conventional methods like membership inference can carry out this detection with some level of accuracy. We show that watermarked training data leaves traces easier to detect and much more reliable than membership inference. We link the contamination level to the watermark robustness, its proportion in the training set, and the fine-tuning process. …

abstract accuracy arxiv cs.ai cs.cl cs.cr cs.lg data detection generated inference language language models llm paper show traces training training data type watermarking