ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline

arXiv:2404.02893v1 Announce Type: new
Abstract: Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many strategies and datasets to enhance LLMs' mathematics are developed, it remains a challenge to simultaneously maintain and improve both language and mathematical capabilities in deployed LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses the challenge in the feedback learning stage of LLM alignment. We first train a general Math-Critique …

arxiv critique cs.cl improving language language models large language large language models math pipeline problem-solving type