VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

arXiv:2405.04950v1 Announce Type: new
Abstract: Large Multimodal Models (LMMs) have achieved impressive success in visual understanding and reasoning, remarkably improving the performance of mathematical reasoning in a visual context. Yet, a challenging type of visual math lies in the multimodal graph theory problem, which demands that LMMs understand the graphical structures accurately and perform multi-step reasoning on the visual graph. Additionally, exploring multimodal graph theory problems will lead to more effective strategies in fields like biology, transportation, and robotics planning. …

abstract arxiv context cs.ai cs.cl cs.cv graph improving large multimodal models lies lmms math mathematical reasoning multimodal multimodal models performance reasoning success theory type understanding visual