Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks

arXiv:2402.11638v1 Announce Type: new
Abstract: The widespread use of large language models (LLMs) is increasing the demand for methods that detect machine-generated text to prevent misuse. The goal of our study is to stress test the detectors' robustness to malicious attacks under realistic scenarios. We comprehensively study the robustness of popular machine-generated text detectors under attacks from diverse categories: editing, paraphrasing, prompting, and co-generating. Our attacks assume limited access to the generator LLMs, and we compare the performance of detectors …

abstract arxiv attacks cs.cl demand generated language language models large language large language models llms machine misuse robustness stress study test testing text type