Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

arXiv:2402.14016v1 Announce Type: new
Abstract: Large Language Models (LLMs) are powerful zero-shot assessors and are increasingly used in real-world situations such as for written exams or benchmarking systems. Despite this, no existing work has analyzed the vulnerability of judge-LLMs against adversaries attempting to manipulate outputs. This work presents the first study on the adversarial robustness of assessment LLMs, where we search for short universal phrases that when appended to texts can deceive LLMs to provide high assessment scores. Experiments on …

abstract adversarial adversarial attacks arxiv assessment attacks benchmarking cs.cl exams judge language language models large language large language models llm llms robust systems type vulnerability work world zero-shot