ViLLM-Eval: A Comprehensive Evaluation Suite for Vietnamese Large Language Models

arXiv:2404.11086v1 Announce Type: new
Abstract: The rapid advancement of large language models (LLMs) necessitates the development of new benchmarks to accurately assess their capabilities. To address this need for Vietnamese, this work aims to introduce ViLLM-Eval, the comprehensive evaluation suite designed to measure the advanced knowledge and reasoning abilities of foundation models within a Vietnamese context. ViLLM-Eval consists of multiple-choice questions and predict next word tasks spanning various difficulty levels and diverse disciplines, ranging from humanities to science and engineering. …

abstract advanced advancement arxiv benchmarks capabilities cs.ai cs.cl development evaluation knowledge language language models large language large language models llms reasoning type work