Exploring the Necessity of Visual Modality in Multimodal Machine Translation using Authentic Datasets

arXiv:2404.06107v1 Announce Type: new
Abstract: Recent research in the field of multimodal machine translation (MMT) has indicated that the visual modality is either dispensable or offers only marginal advantages. However, most of these conclusions are drawn from the analysis of experimental results based on a limited set of bilingual sentence-image pairs, such as Multi30k. In these kinds of datasets, the content of one bilingual parallel sentence pair must be well represented by a manually annotated image, which is different from …

abstract advantages analysis arxiv authentic cs.cl datasets experimental however machine machine translation multimodal research results set translation type visual