A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scene

arXiv:2404.11249v1 Announce Type: new
Abstract: Pre-trained vision-language (V-L) models such as CLIP have shown excellent performance in many downstream cross-modal tasks. However, most of them are only applicable to the English context. Subsequent research has focused on this problem and proposed improved models, such as CN-CLIP and AltCLIP, to facilitate their applicability to Chinese and even other languages. Nevertheless, these models suffer from high latency and a large memory footprint in inference, which limits their further deployment on resource-constrained edge …

abstract alignment arxiv clip context cs.cv distillation english framework however knowledge language modal multilingual performance research tasks them type vision vision-language