all AI news
Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metric. (arXiv:2209.13654v1 [cs.CL])
Sept. 29, 2022, 1:15 a.m. | Giorgos Vernikos, Brian Thompson, Prashant Mathur, Marcello Federico
cs.CL updates on arXiv.org arxiv.org
We hypothesize that existing sentence-level machine translation (MT) metrics
become less effective when the human reference contains ambiguities. To verify
this hypothesis, we present a very simple method for extending pretrained
metrics to incorporate context at the document level. We apply our method to
three popular metrics, BERTScore, Prism, and COMET, and to the reference free
metric COMET-QE. We evaluate the extended metrics on the WMT 2021 metrics
shared task using the provided MQM annotations. Our results show that the …
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Data Scientist (m/f/x/d)
@ Symanto Research GmbH & Co. KG | Spain, Germany
Data Science Sustainability Co-Op (Summer & Fall 2024)
@ O-I | Perrysburg, OH, United States
Research Scientist
@ Chevron Phillips Chemical Company | USA: Kingwood, TX, US, 77339
Data Scientist Python (Django) (m/f/d)
@ RoomPriceGenie | Hybrid Mannheim, Remote DACH, Remote Germany
Operational Transformation & Strategy - Data Operations - Associate
@ JPMorgan Chase & Co. | Mumbai, Maharashtra, India
Senior Data Scientist
@ Rocket Travel | Chicago, IL USA