Assessing Adversarial Robustness of Large Language Models: An Empirical Study

arXiv:2405.02764v1 Announce Type: cross
Abstract: Large Language Models (LLMs) have revolutionized natural language processing, but their robustness against adversarial attacks remains a critical concern. We presents a novel white-box style attack approach that exposes vulnerabilities in leading open-source LLMs, including Llama, OPT, and T5. We assess the impact of model size, structure, and fine-tuning strategies on their resistance to adversarial perturbations. Our comprehensive evaluation across five diverse text classification tasks establishes a new benchmark for LLM robustness. The findings of …

abstract adversarial adversarial attacks arxiv attacks box cs.cl cs.lg impact language language models language processing large language large language models llama llms natural natural language natural language processing novel processing robustness study style type vulnerabilities