Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment. (arXiv:2208.13628v2 [cs.CV] UPDATED)

Vision and Language Pretraining has become the prevalent approach for
tackling multimodal downstream tasks. The current trend is to move towards ever
larger models and pretraining datasets. This computational headlong rush does
not seem reasonable in the long term to move toward sustainable solutions, and
de facto excludes academic laboratories with limited resources. In this work,
we propose a new framework, dubbed ViCHA, that efficiently exploits the input
data to boost the learning by: (a) a new hierarchical cross-modal alignment …

alignment arxiv hierarchical language vision