Multimodal Learning with Channel-Mixing and Masked Autoencoder on Facial Action Unit Detection. (arXiv:2209.12244v1 [cs.CV])

Recent studies utilizing multi-modal data aimed at building a robust model
for facial Action Unit (AU) detection. However, due to the heterogeneity of
multi-modal data, multi-modal representation learning becomes one of the main
challenges. On one hand, it is difficult to extract the relevant features from
multi-modalities by only one feature extractor, on the other hand, previous
studies have not fully explored the potential of multi-modal fusion strategies.
For example, early fusion usually required all modalities to be present during …

arxiv autoencoder detection masked autoencoder multimodal multimodal learning