Concept-skill Transferability-based Data Selection for Large Vision-Language Models

arXiv:2406.10995v1 Announce Type: cross
Abstract: Instruction tuning, or supervised finetuning on extensive task-specific data, is necessary for Large Vision-Language Models (LVLMs) to generalize well across a broad range of vision-language (VL) tasks. However, training on large VL datasets can become prohibitively expensive. In this work, we introduce COINCIDE, an effective and scalable data selection technique that uses a small model as a reference model to select visual instruction tuning data for efficient finetuning of a target LVLM, focusing on diversity …

abstract arxiv become concept cs.cv cs.lg data datasets finetuning however instruction tuning language language models scalable skill tasks training tuning type vision vision-language vision-language models work