Goal-guided Generative Prompt Injection Attack on Large Language Models

arXiv:2404.07234v1 Announce Type: cross
Abstract: Current large language models (LLMs) provide a strong foundation for large-scale user-oriented natural language tasks. A large number of users can easily inject adversarial text or instructions through the user interface, thus causing LLMs model security challenges. Although there is currently a large amount of research on prompt injection attacks, most of these black-box attacks use heuristic strategies. It is unclear how these heuristic strategies relate to the success rate of attacks and thus effectively …

abstract adversarial arxiv challenges cs.ai cs.cl cs.cr current foundation generative language language models large language large language models llms natural natural language prompt prompt injection research scale security security challenges tasks text through type