Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding

arXiv:2403.11311v1 Announce Type: new
Abstract: Deep multimodal semantic understanding that goes beyond the mere superficial content relation mining has received increasing attention in the realm of artificial intelligence. The challenges of collecting and annotating high-quality multi-modal data have underscored the significance of few-shot learning. In this paper, we focus on two critical tasks under this context: few-shot multi-modal sarcasm detection (MSD) and multi-modal sentiment analysis (MSA). To address them, we propose Mixture-of-Prompt-Experts with Block-Aware Prompt Fusion (MoPE-BAF), a novel multi-modal …

abstract artificial artificial intelligence arxiv attention beyond challenges cs.cl cs.mm data experts few-shot few-shot learning focus intelligence mining modal multi-modal multimodal paper prompt quality semantic significance tasks type understanding