Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

arXiv:2404.08517v1 Announce Type: cross
Abstract: While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from multiple aspects, e.g., truthfulness, robustness, and fairness. Recent research has started developing quality assurance methods for LLMs, introducing techniques such as offline detector-based or uncertainty estimation methods. However, these approaches predominantly concentrate on post-generation analysis, leaving the online safety analysis for LLMs during the generation phase an unexplored area. To bridge this gap, …

abstract analysis applications arxiv assessment benchmark concerns cs.ai cs.cl cs.cr cs.lg cs.se fairness fields interpretability language language models large language large language models llms multiple operations path quality quality assurance research robustness safe safety type