Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour Analysis In-the-wild

arXiv:2403.15044v1 Announce Type: new
Abstract: Multimodal fusion is a significant method for most multimodal tasks. With the recent surge in the number of large pre-trained models, combining both multimodal fusion methods and pre-trained model features can achieve outstanding performance in many multimodal tasks. In this paper, we present our approach, which leverages both advantages for addressing the task of Expression (Expr) Recognition and Valence-Arousal (VA) Estimation. We evaluate the Aff-Wild2 database using pre-trained models, then extract the final hidden layers …

abstract analysis arxiv cs.ai cs.cv features fusion multimodal paper performance pre-trained models tasks type