Token-Ensemble Text Generation: On Attacking the Automatic AI-Generated Text Detection

arXiv:2402.11167v1 Announce Type: new
Abstract: The robustness of AI-content detection models against cultivated attacks (e.g., paraphrasing or word switching) remains a significant concern. This study proposes a novel token-ensemble generation strategy to challenge the robustness of current AI-content detection approaches. We explore the ensemble attack strategy by completing the prompt with the next token generated from random candidate LLMs. We find the token-ensemble approach significantly drops the performance of AI-content detection models (The code and test sets will be released). …

abstract ai-content ai-generated text arxiv attacks challenge cs.ai cs.cl current detection ensemble explore generated novel paraphrasing prompt robustness strategy study text text generation the prompt token type word