Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models

arXiv:2405.00402v1 Announce Type: new
Abstract: The alignments of reasoning abilities between smaller and larger Language Models are largely conducted via Supervised Fine-Tuning (SFT) using demonstrations generated from robust Large Language Models (LLMs). Although these approaches deliver more performant models, they do not show sufficiently strong generalization ability as the training only relies on the provided demonstrations.
In this paper, we propose the Self-refine Instruction-tuning method that elicits Smaller Language Models to self-refine their abilities. Our approach is based on a …

abstract arxiv cs.cl fine-tuning generated language language models large language large language models llms reasoning refine robust sft show supervised fine-tuning training type via