RepEval: Effective Text Evaluation with LLM Representation

arXiv:2404.19563v1 Announce Type: new
Abstract: Automatic evaluation metrics for generated texts play an important role in the NLG field, especially with the rapid growth of LLMs. However, existing metrics are often limited to specific scenarios, making it challenging to meet the evaluation requirements of expanding LLM applications. Therefore, there is a demand for new, flexible, and effective metrics. In this study, we introduce RepEval, the first metric leveraging the projection of LLM representations for evaluation. RepEval requires minimal sample pairs …

abstract applications arxiv cs.cl demand evaluation evaluation metrics generated growth however llm llm applications llms making metrics nlg representation requirements role text type