The Case for Evaluating Multimodal Translation Models on Text Datasets

arXiv:2403.03014v1 Announce Type: new
Abstract: A good evaluation framework should evaluate multimodal machine translation (MMT) models by measuring 1) their use of visual information to aid in the translation task and 2) their ability to translate complex sentences such as done for text-only machine translation. However, most current work in MMT is evaluated against the Multi30k testing sets, which do not measure these properties. Namely, the use of visual information by the MMT model cannot be shown directly from the …

abstract arxiv case cs.cl current datasets evaluation framework good information machine machine translation measuring multimodal text translate translation type visual work