Can we do that simpler? Simple, Efficient, High-Quality Evaluation Metrics for NLG. (arXiv:2209.09593v1 [cs.CL])

We explore efficient evaluation metrics for Natural Language Generation
(NLG). To implement efficient metrics, we replace (i) computation-heavy
transformers in metrics such as BERTScore, MoverScore, BARTScore, XMoverScore,
etc. with lighter versions (such as distilled ones) and (ii) cubic inference
time alignment algorithms such as Word Mover Distance with linear and quadratic
approximations. We consider six evaluation metrics (both monolingual and
multilingual), assessed on three different machine translation datasets, and 16
light-weight transformers as replacement. We find, among others, that (a) …

arxiv evaluation metrics nlg quality