FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models

arXiv:2403.07747v1 Announce Type: new
Abstract: To thoroughly assess the mathematical reasoning abilities of Large Language Models (LLMs), we need to carefully curate evaluation datasets covering diverse mathematical concepts and mathematical problems at different difficulty levels. In pursuit of this objective, we propose FineMath in this paper, a fine-grained mathematical evaluation benchmark dataset for assessing Chinese LLMs. FineMath is created to cover the major key mathematical concepts taught in elementary school math, which are further divided into 17 categories of math …

abstract arxiv benchmark chinese concepts cs.ai cs.cl datasets diverse evaluation fine-grained language language models large language large language models llms mathematical reasoning paper reasoning type