Uncovering Safety Risks in Open-source LLMs through Concept Activation Vector

arXiv:2404.12038v1 Announce Type: new
Abstract: Current open-source large language models (LLMs) are often undergone careful safety alignment before public release. Some attack methods have also been proposed that help check for safety vulnerabilities in LLMs to ensure alignment robustness. However, many of these methods have moderate attack success rates. Even when successful, the harmfulness of their outputs cannot be guaranteed, leading to suspicions that these methods have not accurately identified the safety vulnerabilities of LLMs. In this paper, we introduce …

abstract alignment arxiv attack methods check concept cs.cl current however language language models large language large language models llms public release risks robustness safety success through type vector vulnerabilities