SelF-Eval: Self-supervised Fine-grained Dialogue Evaluation. (arXiv:2208.08094v5 [cs.CL] UPDATED)

This paper introduces a novel Self-supervised Fine-grained Dialogue
Evaluation framework (SelF-Eval). The core idea is to model the correlation
between turn quality and the entire dialogue quality. We first propose a novel
automatic data construction method that can automatically assign fine-grained
scores for arbitrarily dialogue data. Then we train \textbf{SelF-Eval} with a
multi-level contrastive learning schema which helps to distinguish different
score levels. Experimental results on multiple benchmarks show that SelF-Eval
is highly consistent with human evaluations and better than …

arxiv evaluation fine-grained