Learning to Watermark LLM-generated Text via Reinforcement Learning

arXiv:2403.10553v1 Announce Type: new
Abstract: We study how to watermark LLM outputs, i.e. embedding algorithmically detectable signals into LLM-generated text to track misuse. Unlike the current mainstream methods that work with a fixed LLM, we expand the watermark design space by including the LLM tuning stage in the watermark pipeline. While prior works focus on token-level watermark that embeds signals into the output, we design a model-level watermark that embeds signals into the LLM weights, and such signals can be …

arxiv cs.ai cs.cr cs.lg generated llm reinforcement reinforcement learning text type via watermark