Generative Multimodal Entity Linking

arXiv:2306.12725v3 Announce Type: replace
Abstract: Multimodal Entity Linking (MEL) is the task of mapping mentions with multimodal contexts to the referent entities from a knowledge base. Existing MEL methods mainly focus on designing complex multimodal interaction mechanisms and require fine-tuning all model parameters, which can be prohibitively costly and difficult to scale in the era of Large Language Models (LLMs). In this work, we propose GEMEL, a Generative Multimodal Entity Linking framework based on LLMs, which directly generates target entity …

abstract arxiv cs.cl designing fine-tuning focus generative knowledge knowledge base mapping multimodal parameters scale type