[R] Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

**Paper**: [https://arxiv.org/abs/2401.01335](https://arxiv.org/abs/2401.01335)

**Abstract**:

>Harnessing the power of human-annotated data through Supervised Fine-Tuning (SFT) is pivotal for advancing Large Language Models (LLMs). In this paper, we delve into the prospect of growing a strong LLM out of a weak one without the need for acquiring additional human-annotated data. We propose a new fine-tuning method called Self-Play fIne-tuNing (**SPIN**), which starts from a supervised fine-tuned model. At the heart of SPIN lies a self-play mechanism, where the LLM refines its capability by …

abstract annotated data data fine-tuning human language language models large language large language models llm llms machinelearning paper pivotal power self-play sft spin supervised fine-tuning through