Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning

arXiv:2403.02333v1 Announce Type: new
Abstract: Large language models (LLMs) have shown great potential in complex reasoning tasks, yet their performance is often hampered by the scarcity of high-quality, reasoning-focused training datasets. Addressing this challenge, we propose Key-Point-Driven Data Synthesis (KPDDS), a novel data synthesis framework that synthesizes question-answer pairs by leveraging key points and exemplar pairs from authentic data sources. KPDDS ensures the generation of novel questions with rigorous quality control and substantial scalability. As a result, we present KPMath, …

abstract arxiv challenge cs.ai cs.cl data datasets driven data framework key language language models large language large language models llms mathematical reasoning novel performance quality question reasoning synthesis tasks training training datasets type