Open-Domain Text Evaluation via Contrastive Distribution Modeling

arXiv:2306.11879v2 Announce Type: replace
Abstract: Recent advancements in open-domain text generation, driven by the power of large pre-trained language models (LLMs), have demonstrated remarkable performance. However, assessing these models' generation quality remains a challenge. In this paper, we introduce a novel method for evaluating open-domain text generation called Contrastive Distribution Methods (CDM). Leveraging the connection between increasing model parameters and enhanced LLM performance, CDM creates a mapping from the _contrast_ of two probabilistic distributions -- one known to be superior …

abstract arxiv challenge cs.cl distribution domain evaluation however language language models llms modeling novel paper performance power quality text text generation type via