Embedded Heterogeneous Attention Transformer for Cross-lingual Image Captioning

arXiv:2307.09915v2 Announce Type: replace
Abstract: Cross-lingual image captioning is a challenging task that requires addressing both cross-lingual and cross-modal obstacles in multimedia analysis. The crucial issue in this task is to model the global and the local matching between the image and different languages. Existing cross-modal embedding methods based on the transformer architecture oversee the local matching between the image region and monolingual words, especially when dealing with diverse languages. To overcome these limitations, we propose an Embedded Heterogeneous Attention …

abstract analysis arxiv attention captioning cross-lingual cs.cv cs.mm embedded embedding global image issue languages modal multimedia obstacles transformer type