BMX: Boosting Natural Language Generation Metrics with Explainability

arXiv:2212.10469v2 Announce Type: replace
Abstract: State-of-the-art natural language generation evaluation metrics are based on black-box language models. Hence, recent works consider their explainability with the goals of better understandability for humans and better metric analysis, including failure cases. In contrast, our proposed method BMX: Boosting Natural Language Generation Metrics with explainability explicitly leverages explanations to boost the metrics' performance. In particular, we perceive feature importance explanations as word-level scores, which we convert, via power means, into a segment-level score. We …

abstract analysis art arxiv boosting box cases contrast cs.cl evaluation evaluation metrics explainability failure humans language language generation language models metrics natural natural language natural language generation state type