Variational Multi-Modal Hypergraph Attention Network for Multi-Modal Relation Extraction

arXiv:2404.12006v1 Announce Type: new
Abstract: Multi-modal relation extraction (MMRE) is a challenging task that aims to identify relations between entities in text leveraging image information. Existing methods are limited by their neglect of the multiple entity pairs in one sentence sharing very similar contextual information (ie, the same text and image), resulting in increased difficulty in the MMRE task. To address this limitation, we propose the Variational Multi-Modal Hypergraph Attention Network (VM-HAN) for multi-modal relation extraction. Specifically, we first construct …

abstract arxiv attention cs.cl extraction hypergraph identify image information modal multi-modal multiple network relations text type