The Power of Noise: Toward a Unified Multi-modal Knowledge Graph Representation Framework

arXiv:2403.06832v1 Announce Type: new
Abstract: The advancement of Multi-modal Pre-training highlights the necessity for a robust Multi-Modal Knowledge Graph (MMKG) representation learning framework. This framework is crucial for integrating structured knowledge into multi-modal Large Language Models (LLMs) at scale, aiming to alleviate issues like knowledge misconceptions and multi-modal hallucinations. In this work, to evaluate models' ability to accurately embed entities within MMKGs, we focus on two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA). Building …

arxiv cs.ai cs.cl framework graph graph representation knowledge knowledge graph modal multi-modal noise power representation type