Toxicity in Multilingual Machine Translation at Scale. (arXiv:2210.03070v1 [cs.CL])

Machine Translation systems can produce different types of errors, some of
which get characterized as critical or catastrophic due to the specific
negative impact they can have on users. Automatic or human evaluation metrics
do not necessarily differentiate between such critical errors and more
innocuous ones. In this paper we focus on one type of critical error: added
toxicity. We evaluate and analyze added toxicity when translating a large
evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic
axes) from …

arxiv machine machine translation scale toxicity translation