Automatic Answerability Evaluation for Question Generation

arXiv:2309.12546v2 Announce Type: replace
Abstract: Conventional automatic evaluation metrics, such as BLEU and ROUGE, developed for natural language generation (NLG) tasks, are based on measuring the n-gram overlap between the generated and reference text. These simple metrics may be insufficient for more complex tasks, such as question generation (QG), which requires generating questions that are answerable by the reference answers. Developing a more sophisticated automatic evaluation metric, thus, remains an urgent problem in QG research. This work proposes PMAN (Prompting-based …

abstract arxiv bleu cs.cl evaluation evaluation metrics generated language language generation measuring metrics natural natural language natural language generation nlg question questions reference simple tasks text type