MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

arXiv:2403.14624v1 Announce Type: cross
Abstract: The remarkable progress of Multi-modal Large Language Models (MLLMs) has garnered unparalleled attention, due to their superior performance in visual contexts. However, their capabilities in visual math problem-solving remain insufficiently evaluated and understood. We investigate current benchmarks to incorporate excessive visual content within textual questions, which potentially assist MLLMs in deducing answers without truly interpreting the input diagrams. To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth …

arxiv cs.ai cs.cl cs.cv cs.lg diagrams llm math modal multi-modal type visual