all AI news
Distribution Aware Metrics for Conditional Natural Language Generation. (arXiv:2209.07518v2 [cs.CL] UPDATED)
cs.CL updates on arXiv.org arxiv.org
Traditional automated metrics for evaluating conditional natural language
generation use pairwise comparisons between a single generated text and the
best-matching gold-standard ground truth text. When multiple ground truths are
available, scores are aggregated using an average or max operation across
references. While this approach works well when diversity in the ground truth
data (i.e. dispersion of the distribution of conditional texts) can be ascribed
to noise, such as in automated speech recognition, it does not allow for robust
evaluation in …
arxiv distribution language language generation metrics natural natural language natural language generation