Red Teaming Game: A Game-Theoretic Framework for Red Teaming Language Models

arXiv:2310.00322v3 Announce Type: replace
Abstract: Deployable Large Language Models (LLMs) must conform to the criterion of helpfulness and harmlessness, thereby achieving consistency between LLMs outputs and human values. Red-teaming techniques constitute a critical way towards this criterion. Existing work rely solely on manual red team designs and heuristic adversarial prompts for vulnerability detection and optimization. These approaches lack rigorous mathematical formulation, thus limiting the exploration of diverse attack strategy within quantifiable measure and optimization of LLMs under convergence guarantees. In …

abstract adversarial arxiv criterion cs.cl cs.gt designs framework game human language language models large language large language models llms red team red teaming team type values work