AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages

arXiv:2311.09828v2 Announce Type: replace
Abstract: Despite the recent progress on scaling multilingual machine translation (MT) to several under-resourced African languages, accurately measuring this progress remains challenging, since evaluation is often performed on n-gram matching metrics such as BLEU, which typically show a weaker correlation with human judgments. Learned metrics such as COMET have higher correlation; however, the lack of evaluation data with human ratings for under-resourced languages, complexity of annotation guidelines like Multidimensional Quality Metrics (MQM), and limited language coverage …

abstract arxiv bleu comet correlation cs.cl evaluation human languages machine machine translation measuring metrics multilingual progress scaling show translation type