Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling

arXiv:2307.14126v2 Announce Type: replace
Abstract: The missing modality issue is critical but non-trivial to be solved by multi-modal models. Current methods aiming to handle the missing modality problem in multi-modal tasks, either deal with missing modalities only during evaluation or train separate models to handle specific missing modality settings. In addition, these models are designed for specific tasks, so for example, classification models are not easily adapted to segmentation tasks and vice versa. In this paper, we propose the Shared-Specific …

arxiv cs.cv feature modal modelling multi-modal replace type via