MMCode: Evaluating Multi-Modal Code Large Language Models with Visually Rich Programming Problems

arXiv:2404.09486v1 Announce Type: cross
Abstract: Programming often involves converting detailed and complex specifications into code, a process during which developers typically utilize visual aids to more effectively convey concepts. While recent developments in Large Multimodal Models have demonstrated remarkable abilities in visual reasoning and mathematical tasks, there is little work on investigating whether these models can effectively interpret visual elements for code generation. To this end, we present MMCode, the first multi-modal coding dataset for evaluating algorithmic problem-solving skills in …

arxiv code cs.cl cs.cv cs.se language language models large language large language models modal multi-modal programming type