Small Language Models Need Strong Verifiers to Self-Correct Reasoning

arXiv:2404.17140v1 Announce Type: new
Abstract: Self-correction has emerged as a promising solution to boost the reasoning performance of large language models (LLMs), where LLMs refine their solutions using self-generated critiques that pinpoint the errors. This work explores whether smaller-size (<= 13B) language models (LMs) have the ability of self-correction on reasoning tasks with minimal inputs from stronger LMs. We propose a novel pipeline that prompts smaller LMs to collect self-correction data that supports the training of self-refinement abilities. First, we …

13b abstract arxiv boost cs.cl errors generated language language models large language large language models llms lms performance reasoning refine small small language models solution solutions type work