TFusion: Transformer based N-to-One Multimodal Fusion Block. (arXiv:2208.12776v1 [cs.CV])

People perceive the world with different senses, such as sight, hearing,
smell, and touch. Processing and fusing information from multiple modalities
enables Artificial Intelligence to understand the world around us more easily.
However, when there are missing modalities, the number of available modalities
is different in diverse situations, which leads to an N-to-One fusion problem.
To solve this problem, we propose a transformer based fusion block called
TFusion. Different from preset formulations or convolution based methods, the
proposed block automatically …

arxiv cv fusion multimodal transformer