Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese

arXiv:2404.07824v1 Announce Type: new
Abstract: Vision Language Models (VLMs) have undergone a rapid evolution, giving rise to significant advancements in the realm of multimodal understanding tasks. However, the majority of these models are trained and evaluated on English-centric datasets, leaving a gap in the development and evaluation of VLMs for other languages, such as Japanese. This gap can be attributed to the lack of methodologies for constructing VLMs and the absence of benchmarks to accurately measure their performance. To address …

abstract arxiv benchmark cs.cl cs.cv datasets development english evaluation evolution gap giving however japanese language language models multimodal realm tasks type understanding vision vlms