Quality Estimation with $k$-nearest Neighbors and Automatic Evaluation for Model-specific Quality Estimation

arXiv:2404.18031v1 Announce Type: new
Abstract: Providing quality scores along with Machine Translation (MT) output, so-called reference-free Quality Estimation (QE), is crucial to inform users about the reliability of the translation. We propose a model-specific, unsupervised QE approach, termed $k$NN-QE, that extracts information from the MT model's training data using $k$-nearest neighbors. Measuring the performance of model-specific QE is not straightforward, since they provide quality scores on their own MT output, thus cannot be evaluated using benchmark QE test sets containing …

abstract arxiv cs.cl data evaluation free information machine machine translation neighbors quality reference reliability training training data translation type unsupervised