MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs

arXiv:2402.16352v1 Announce Type: new
Abstract: Large language models (LLMs) have exhibited great potential in mathematical reasoning. However, there remains a performance gap in this area between existing open-source models and closed-source models such as GPT-4. In this paper, we introduce MathGenie, a novel method for generating diverse and reliable math problems from a small-scale problem-solution dataset (denoted as seed data). We augment the ground-truth solutions of our seed data and train a back-translation model to translate the augmented solutions back …

abstract arxiv cs.ai cs.cl data diverse gap gpt gpt-4 language language models large language large language models llms mathematical reasoning novel open-source models paper performance question reasoning synthetic synthetic data translation type