[R] A Careful Examination of Large Language Model Performance on Grade School Arithmetic

**Paper**: [https://arxiv.org/abs/2405.00332](https://arxiv.org/abs/2405.00332)

**Abstract**:

>Large language models (LLMs) have achieved impressive success on many benchmarks for mathematical reasoning. However, there is growing concern that some of this performance actually reflects dataset contamination, where data closely resembling benchmark questions leaks into the training data, instead of true reasoning ability. To investigate this claim rigorously, we commission ***Grade School Math 1000*** (**GSM1k**). GSM1k is designed to mirror the style and complexity of the established GSM8k benchmark, the gold standard for measuring elementary mathematical …

abstract benchmark benchmarks claim commission data dataset however language language models large language large language models leaks llms machinelearning math mathematical reasoning performance questions reasoning school success training training data true