Exploring Precision and Recall to assess the quality and diversity of LLMs

arXiv:2402.10693v1 Announce Type: cross
Abstract: This paper introduces a novel evaluation framework for Large Language Models (LLMs) such as Llama-2 and Mistral, focusing on the adaptation of Precision and Recall metrics from image generation to text generation. This approach allows for a nuanced assessment of the quality and diversity of generated text without the need for aligned corpora. By conducting a comprehensive evaluation of state-of-the-art language models, the study reveals significant insights into their performance on open-ended generation tasks, which …

abstract arxiv assessment cs.cl cs.lg diversity evaluation framework image image generation language language models large language large language models llama llms metrics mistral novel paper precision quality recall text text generation type