Well begun is half done: Importance of Starting Right in Multi-Step Math Reasoning

arXiv:2311.07945v2 Announce Type: replace
Abstract: Smaller language models can solve complex reasoning tasks better by learning to generate rationales for their predictions. However, we observe that these smaller models can sometimes struggle to start correctly, but when corrected, can solve a task that they would otherwise have struggled with. We propose two ways in which a smaller model can benefit from initial guidance: 1) asking an LLM for initial guidance, and 2) self-questioning guidance, where the student model can first …

abstract arxiv begun cs.cl generate importance language language models math observe predictions reasoning solve struggle tasks type