all AI news
Advancing Geometric Problem Solving: A Comprehensive Benchmark for Multimodal Model Evaluation
April 9, 2024, 4:50 a.m. | Kai Sun, Yushi Bai, Nianyi Lin
cs.CL updates on arXiv.org arxiv.org
Abstract: In this work, we present the MM-MATH dataset, a novel benchmark developed to rigorously evaluate the performance of advanced large language and multimodal models - including but not limited to GPT-4, GPT-4V, and Claude - within the domain of geometric computation. This dataset comprises 5,929 meticulously crafted geometric problems, each paired with a corresponding image, aimed at mirroring the complexity and requirements typical of ninth-grade mathematics. The motivation behind MM-MATH stems from the burgeoning interest …
abstract advanced arxiv benchmark claude computation cs.cl dataset domain evaluation gpt gpt-4 gpt-4v language large language math multimodal multimodal model multimodal models novel performance type work
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Robotics Technician - 3rd Shift
@ GXO Logistics | Perris, CA, US, 92571