Curiosity-driven Red-teaming for Large Language Models

arXiv:2402.19464v1 Announce Type: new
Abstract: Large language models (LLMs) hold great potential for many natural language applications but risk generating incorrect or toxic content. To probe when an LLM generates unwanted content, the current paradigm is to recruit a \textit{red team} of human testers to design input prompts (i.e., test cases) that elicit undesirable responses from LLMs. However, relying solely on human testers is expensive and time-consuming. Recent works automate red teaming by training a separate red team LLM with …

arxiv cs.ai cs.cl cs.lg curiosity language language models large language large language models type