On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning

arXiv:2404.02254v1 Announce Type: cross
Abstract: In multimodal machine learning, multiple modalities of data (e.g., text and images) are combined to facilitate the learning of a better machine learning model, which remains applicable to a corresponding unimodal task (e.g., text generation). Recently, multimodal machine learning has enjoyed huge empirical success (e.g. GPT-4). Motivated to develop theoretical justification for this empirical success, Lu (NeurIPS '23, ALT '24) introduces a theory of multimodal learning, and considers possible separations between theoretical models of multimodal …

abstract arxiv computational cs.lg data images machine machine learning machine learning model multimodal multiple stat.ml success text text generation type