all AI news
DeltaScore: Fine-Grained Story Evaluation with Perturbations. (arXiv:2303.08991v5 [cs.CL] UPDATED)
cs.CL updates on arXiv.org arxiv.org
Numerous evaluation metrics have been developed for natural language
generation tasks, but their effectiveness in evaluating stories is limited as
they are not specifically tailored to assess intricate aspects of storytelling,
such as fluency and interestingness. In this paper, we introduce DELTASCORE, a
novel methodology that employs perturbation techniques for the evaluation of
nuanced story aspects. Our central proposition posits that the extent to which
a story excels in a specific aspect (e.g., fluency) correlates with the
magnitude of its …
arxiv evaluation evaluation metrics fine-grained language language generation methodology metrics natural natural language natural language generation novel paper stories story storytelling tasks