FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning

arXiv:2404.15182v1 Announce Type: new
Abstract: In the rapidly evolving field of artificial intelligence, multimodal models, e.g., integrating vision and language into visual-language models (VLMs), have become pivotal for many applications, ranging from image captioning to multimodal search engines. Among these models, the Contrastive Language-Image Pre-training (CLIP) model has demonstrated remarkable performance in understanding and generating nuanced relationships between text and images. However, the conventional training of such models often requires centralized aggregation of vast datasets, posing significant privacy and data …

abstract applications artificial artificial intelligence arxiv become captioning clip cs.ai cs.lg federated learning flora image intelligence language language models multimodal multimodal models performance pivotal pre-training search training type vision vision-language vision-language models visual vlms