Advancing Geometric Problem Solving: A Comprehensive Benchmark for Multimodal Model Evaluation

arXiv:2404.05091v1 Announce Type: new
Abstract: In this work, we present the MM-MATH dataset, a novel benchmark developed to rigorously evaluate the performance of advanced large language and multimodal models - including but not limited to GPT-4, GPT-4V, and Claude - within the domain of geometric computation. This dataset comprises 5,929 meticulously crafted geometric problems, each paired with a corresponding image, aimed at mirroring the complexity and requirements typical of ninth-grade mathematics. The motivation behind MM-MATH stems from the burgeoning interest …

abstract advanced arxiv benchmark claude computation cs.cl dataset domain evaluation gpt gpt-4 gpt-4v language large language math multimodal multimodal model multimodal models novel performance type work