Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models

arXiv:2309.15098v2 Announce Type: replace
Abstract: We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the LLM interacts internally with factual constraints. We find a strong positive relationship between the LLM's attention to constraint tokens and the factual accuracy of generations. We curate a suite of 10 datasets containing over 40,000 prompts to study the task of …

abstract arxiv attention behavior constraints cs.ai cs.cl cs.lg errors framework generate language language models large language large language models lens llm llms modeling queries text transformer type