CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation

arXiv:2311.18702v2 Announce Type: replace
Abstract: Since the natural language processing (NLP) community started to make large language models (LLMs) act as a critic to evaluate the quality of generated texts, most of the existing works train a critique generation model on the evaluation data labeled by GPT-4's direct prompting. We observe that these models lack the ability to generate informative critiques in both pointwise grading and pairwise comparison especially without references. As a result, their generated critiques cannot provide fine-grained …

abstract act arxiv community critique cs.ai cs.cl data evaluation generated language language model language models language processing large language large language model large language models llms natural natural language natural language processing nlp processing quality replace train type