Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport

arXiv:2401.15489v2 Announce Type: replace
Abstract: Deep learning models for multimodal expression recognition have reached remarkable performance in controlled laboratory environments because of their ability to learn complementary and redundant semantic information. However, these models struggle in the wild, mainly because of the unavailability and quality of modalities used for training. In practice, only a subset of the training-time modalities may be available at test time. Learning with privileged information enables models to exploit data from additional modalities that are only …

abstract arxiv cs.ai cs.cv deep learning environments however information laboratory learn multimodal performance quality recognition semantic struggle training transport type