Unsupervised Multimodal Language Representations using Convolutional Autoencoders. (arXiv:2110.03007v2 [cs.CL] UPDATED)

Multimodal Language Analysis is a demanding area of research, since it is
associated with two requirements: combining different modalities and capturing
temporal information. During the last years, several works have been proposed
in the area, mostly centered around supervised learning in downstream tasks. In
this paper we propose extracting unsupervised Multimodal Language
representations that are universal and can be applied to different tasks.
Towards this end, we map the word-level aligned multimodal sequences to 2-D
matrices and then use Convolutional …

arxiv language multimodal unsupervised