all AI news
Is Context Helpful for Chat Translation Evaluation?
March 14, 2024, 4:48 a.m. | Sweta Agrawal, Amin Farajian, Patrick Fernandes, Ricardo Rei, Andr\'e F. T. Martins
cs.CL updates on arXiv.org arxiv.org
Abstract: Despite the recent success of automatic metrics for assessing translation quality, their application in evaluating the quality of machine-translated chats has been limited. Unlike more structured texts like news, chat conversations are often unstructured, short, and heavily reliant on contextual information. This poses questions about the reliability of existing sentence-level metrics in this domain as well as the role of context in assessing the translation quality. Motivated by this, we conduct a meta-evaluation of existing …
abstract application arxiv chat context conversations cs.cl evaluation information machine metrics quality questions reliability success translated translation type unstructured
More from arxiv.org / cs.CL updates on arXiv.org
Benchmarking LLMs via Uncertainty Quantification
2 days, 5 hours ago |
arxiv.org
CARE: Extracting Experimental Findings From Clinical Literature
2 days, 5 hours ago |
arxiv.org
Jobs in AI, ML, Big Data
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Software Engineering Manager, Generative AI - Characters
@ Meta | Bellevue, WA | Menlo Park, CA | Seattle, WA | New York City | San Francisco, CA
Senior Operations Research Analyst / Predictive Modeler
@ LinQuest | Colorado Springs, Colorado, United States