ANGO: A Next-Level Evaluation Benchmark For Generation-Oriented Language Models In Chinese Domain

arXiv:2401.04898v2 Announce Type: replace
Abstract: Recently, various Large Language Models (LLMs) evaluation datasets have emerged, but most of them have issues with distorted rankings and difficulty in model capabilities analysis. Addressing these concerns, this paper introduces ANGO, a Chinese multi-choice question evaluation benchmark. ANGO proposes Keypoint categorization standard for the first time, each question in ANGO can correspond to multiple keypoints, effectively enhancing interpretability of evaluation results. Base on performance of real humans, we build a quantifiable question difficulty standard …

abstract analysis arxiv benchmark capabilities chinese concerns cs.ai cs.cl datasets domain evaluation language language models large language large language models llms next paper question rankings standard them type