Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information

arXiv:2311.11509v3 Announce Type: replace-cross
Abstract: In recent years, Large Language Models (LLM) have emerged as pivotal tools in various applications. However, these models are susceptible to adversarial prompt attacks, where attackers can carefully curate input strings that mislead LLMs into generating incorrect or undesired outputs. Previous work has revealed that with relatively simple yet effective attacks based on discrete optimization, it is possible to generate adversarial prompts that bypass moderation and alignment of the models. This vulnerability to adversarial prompts …

abstract adversarial applications arxiv attacks cs.cl cs.lg detection information language language models large language large language models llm llms perplexity pivotal prompt strings token tools type work