Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey

arXiv:2302.10035v3 Announce Type: replace
Abstract: With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc. Inspired by the success of these models in single domains (like computer vision and natural language processing), the multi-modal pre-trained big models have also drawn more and more attention in recent years. In this work, we give a comprehensive survey of these models and hope this paper could provide new insights and helps fresh researchers …

arxiv cs.ai cs.cv cs.mm modal multi-modal pre-trained models scale survey type