[R] Is self-correction a viable method to improve LLM reasoning? Probably not.

Can LLMs actually improve their own reasoning by self-correcting mistakes? A new paper from DeepMind and the University of Illinois looks to answer this quantitatively.

The results show that unaided, LLMs struggle at self-correction for reasoning tasks. The core issue is LLMs have trouble reliably evaluating the correctness of their own responses. They rarely identify flaws in initial reasoning. Sometimes LLMs even alter initially correct responses to become incorrect after self-correction! *(I've personally seen this when interacting with ChatGPT many …

core deepmind illinois issue llm llm reasoning llms machinelearning mistakes paper reasoning show tasks university