Dissociation of Faithful and Unfaithful Reasoning in LLMs

arXiv:2405.15092v1 Announce Type: cross
Abstract: Large language models (LLMs) improve their performance in downstream tasks when they generate Chain of Thought reasoning text before producing an answer. Our research investigates how LLMs recover from errors in Chain of Thought, reaching the correct final answer despite mistakes in the reasoning text. Through analysis of these error recovery behaviors, we find evidence for unfaithfulness in Chain of Thought, but we also identify many clear examples of faithful error recovery behaviors. We identify …

abstract analysis arxiv chain of thought cs.ai cs.cl errors generate language language models large language large language models llms mistakes performance reasoning research tasks text thought through type