Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

arXiv:2406.14546v1 Announce Type: new
Abstract: One way to address safety risks from large language models (LLMs) is to censor dangerous knowledge from their training data. While this removes the explicit information, implicit information can remain scattered across various training documents. Could an LLM infer the censored knowledge by piecing together these implicit hints? As a step towards answering this question, we study inductive out-of-context reasoning (OOCR), a type of generalization in which LLMs infer latent information from evidence distributed across …

abstract arxiv cs.ai cs.cl cs.lg data documents information knowledge language language models large language large language models llm llms risks safety safety risks training training data type while