Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition

arXiv:2404.10904v1 Announce Type: new
Abstract: Human communication is multi-modal; e.g., face-to-face interaction involves auditory signals (speech) and visual signals (face movements and hand gestures). Hence, it is essential to exploit multiple modalities when designing machine learning-based facial expression recognition systems. In addition, given the ever-growing quantities of video data that capture human facial expressions, such systems should utilize raw unlabeled videos without requiring expensive annotations. Therefore, in this work, we employ a multitask multi-modal self-supervised learning method for facial expression …

abstract arxiv communication cs.cv data designing ever exploit face facial expression gestures human machine machine learning modal movements multi-modal multiple recognition self-supervised learning speech supervised learning systems type video video data visual