Multimodal Multi-loss Fusion Network for Sentiment Analysis

arXiv:2308.00264v3 Announce Type: replace-cross
Abstract: This paper investigates the optimal selection and fusion of feature encoders across multiple modalities and combines these in one neural network to improve sentiment detection. We compare different fusion methods and examine the impact of multi-loss training within the multi-modality fusion network, identifying surprisingly important findings relating to subnet performance. We have also found that integrating context significantly enhances model performance. Our best model achieves state-of-the-art performance for three datasets (CMU-MOSI, CMU-MOSEI and CH-SIMS). These …

abstract analysis arxiv cs.ai cs.cl cs.lg cs.mm detection feature fusion impact loss multimodal multiple network neural network paper sentiment sentiment analysis training type