Supervised Visual Attention for Simultaneous Multimodal Machine Translation. (arXiv:2201.09324v2 [cs.CL] UPDATED)

Recently, there has been a surge in research in multimodal machine
translation (MMT), where additional modalities such as images are used to
improve translation quality of textual systems. A particular use for such
multimodal systems is the task of simultaneous machine translation, where
visual context has been shown to complement the partial information provided by
the source sentence, especially in the early phases of translation. In this
paper, we propose the first Transformer-based simultaneous MMT architecture,
which has not been …

arxiv attention machine machine translation multimodal translation visual attention