Web: http://arxiv.org/abs/2208.11646

Sept. 16, 2022, 1:16 a.m. | Cyril Chhun, Pierre Colombo, Chloé Clavel, Fabian M. Suchanek

cs.CL updates on arXiv.org arxiv.org

Research on Automatic Story Generation (ASG) relies heavily on human and
automatic evaluation. However, there is no consensus on which human evaluation
criteria to use, and no analysis of how well automatic criteria correlate with
them. In this paper, we propose to re-evaluate ASG evaluation. We introduce a
set of 6 orthogonal and comprehensive human criteria, carefully motivated by
the social sciences literature. We also present HANNA, an annotated dataset of
1,056 stories produced by 10 different ASG systems. HANNA …

arxiv benchmark evaluation human metrics

More from arxiv.org / cs.CL updates on arXiv.org

Postdoctoral Fellow: ML for autonomous materials discovery

@ Lawrence Berkeley National Lab | Berkeley, CA

Research Scientists

@ ODU Research Foundation | Norfolk, Virginia

Embedded Systems Engineer (Robotics)

@ Neo Cybernetica | Bedford, New Hampshire

2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

@ Lawrence Berkeley National Lab | San Francisco, CA

Senior Manager Data Scientist

@ NAV | Remote, US

Senior AI Research Scientist

@ Earth Species Project | Remote anywhere

Research Fellow- Center for Security and Emerging Technology (Multiple Opportunities)

@ University of California Davis | Washington, DC

Staff Fellow - Data Scientist

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Staff Fellow - Senior Data Engineer

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Research Engineer - VFX, Neural Compositing

@ Flawless | Los Angeles, California, United States

[Job-TB] Senior Data Engineer

@ CI&T | Brazil

Data Analytics Engineer

@ The Fork | Paris, France