CheckEval: Robust Evaluation Framework using Large Language Model via Checklist

arXiv:2403.18771v1 Announce Type: new
Abstract: We introduce CheckEval, a novel evaluation framework using Large Language Models, addressing the challenges of ambiguity and inconsistency in current evaluation methods. CheckEval addresses these challenges by dividing evaluation criteria into detailed sub-aspects and constructing a checklist of Boolean questions for each, simplifying the evaluation. This approach not only renders the process more interpretable but also significantly enhances the robustness and reliability of results by focusing on specific evaluation dimensions. Validated through a focused case …

abstract arxiv challenges checklist cs.cl current evaluation framework language language model language models large language large language model large language models novel questions robust simplifying type via