[R] Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

**Paper**: [https://arxiv.org/abs/2312.06585](https://arxiv.org/abs/2312.06585)

**Abstract**:

>Fine-tuning language models\~(LMs) on human-generated data remains a prevalent practice. However, the performance of such models is often limited by the quantity and diversity of high-quality human data. In this paper, we explore whether we can go beyond human data on tasks where we have access to scalar feedback, for example, on math problems where one can verify correctness. To do so, we investigate a simple self-training method based on expectation-maximization, which we call **ReST*****^(EM)***, where we …

abstract beyond data diversity example explore feedback fine-tuning generated human language language models machinelearning math paper performance practice quality tasks verify