The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning

arXiv:2403.03218v1 Announce Type: new
Abstract: The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and chemical weapons. To measure these risks of malicious use, government institutions and major AI labs are developing evaluations for hazardous capabilities in LLMs. However, current evaluations are private, preventing further research into mitigating risk. Furthermore, they focus on only a few, highly specific pathways for malicious use. To fill these gaps, …

arxiv benchmark cs.ai cs.cl cs.cy cs.lg measuring type unlearning