Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References

arXiv:2305.15067v2 Announce Type: replace
Abstract: Most research about natural language generation (NLG) relies on evaluation benchmarks with limited references for a sample, which may result in poor correlations with human judgements. The underlying reason is that one semantic meaning can actually be expressed in different forms, and the evaluation with a single or few references may not accurately reflect the quality of the model's hypotheses. To address this issue, this paper presents a simple and effective method, named Div-Ref, to …

arxiv cs.cl evaluation improving metrics nlg type