Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models

arXiv:2303.13809v3 Announce Type: replace
Abstract: Generative large language models (LLMs), e.g., ChatGPT, have demonstrated remarkable proficiency across several NLP tasks, such as machine translation, text summarization. Recent research (Kocmi and Federmann, 2023) has shown that utilizing LLMs for assessing the quality of machine translation (MT) achieves state-of-the-art performance at the system level but \textit{performs poorly at the segment level}. To further improve the performance of LLMs on MT quality assessment, we investigate several prompting designs, and propose a new prompting …

abstract analysis art arxiv chatgpt cs.cl error evaluation generative human human-like language language models large language large language models llms machine machine translation nlp performance prompting quality research state summarization tasks text text summarization translation type