Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning

arXiv:2305.08473v2 Announce Type: replace-cross
Abstract: Designing an effective representation learning method for multimodal sentiment analysis tasks is a crucial research direction. The challenge lies in learning both shared and private information in a complete modal representation, which is difficult with uniform multimodal labels and a raw feature fusion approach. In this work, we propose a deep modal shared information learning module based on the covariance matrix to capture the shared information between modalities. Additionally, we use a label generation module …

abstract alignment analysis arxiv challenge cs.cl cs.cv designing information labels lies modal multimodal multi-task learning representation representation learning research sentiment sentiment analysis tasks type uniform