[R] Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks - Microsoft 2022 - Many SOTA results spanning vision and vision-language tasks / Multimodal foundation model.

Paper: [https://arxiv.org/abs/2208.10442](https://arxiv.org/abs/2208.10442)

Github: [https://github.com/microsoft/unilm/tree/master/beit](https://github.com/microsoft/unilm/tree/master/beit) Code will be released here. I only found the link on [paperswithcode.com](https://paperswithcode.com) !

Abstract:

>A big convergence of language, vision, and multimodal pretraining is emerging. In this work, we introduce a general-purpose **multimodal foundation model** **BEiT-3**, which achieves **state-of-the-art transfer performance** on both vision and vision-language tasks. Specifically, we advance the big convergence from three aspects: **backbone architecture, pretraining task, and model scaling up**. We introduce Multiway Transformers for **general-purpose modeling**, where the modular architecture enables …

foreign language foundation model image language machinelearning microsoft multimodal sota vision