VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?

arXiv:2404.05955v1 Announce Type: new
Abstract: Multimodal Large Language models (MLLMs) have shown promise in web-related tasks, but evaluating their performance in the web domain remains a challenge due to the lack of comprehensive benchmarks. Existing benchmarks are either designed for general multimodal tasks, failing to capture the unique characteristics of web pages, or focus on end-to-end web agent tasks, unable to measure fine-grained abilities such as OCR, understanding, and grounding. In this paper, we introduce \bench{}, a multimodal benchmark designed …

abstract arxiv benchmarks challenge cs.ai cs.cl domain general language language models large language large language models llms mllms multimodal multimodal llms page performance tasks type understanding web