Interpretable Tensor Fusion

arXiv:2405.04671v1 Announce Type: new
Abstract: Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce interpretable tensor fusion (InTense), a multimodal learning method for training neural networks to simultaneously learn multimodal data representations and their interpretable fusion. InTense can separately capture both linear combinations and multiplicative interactions of diverse data types, thereby disentangling higher-order interactions from the …

abstract applications arxiv audio cs.lg data diverse fusion however images learn machine machine learning multimodal multimodal data multimodal learning networks neural networks practical tensor text training type types