NL-ITI: Optimizing Probing and Intervention for Improvement of ITI Method

arXiv:2403.18680v1 Announce Type: cross
Abstract: Large Language Models (LLM) are prone to returning false information. It constitutes one of major challenges in the AI field. In our work, we explore paradigm introduced by Inference-Time-Intervention (ITI). In first stage, it identifies attention heads, which contain the highest amount of desired type of knowledge (e.g., truthful). Afterwards, during inference, LLM activations are shifted for chosen subset of attention heads. We further improved the ITI framework by introducing a nonlinear probing and multi-token …

abstract arxiv attention challenges cs.cl cs.lg explore false improvement inference information language language models large language large language models llm major paradigm stage type work