VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness

arXiv:2401.07853v2 Announce Type: replace
Abstract: Finetuning a pretrained vision model (PVM) is a common technique for learning downstream vision tasks. However, the conventional finetuning process with randomly sampled data points results in diminished training efficiency. To address this drawback, we propose a novel approach, Vision-language Collaborative Active Finetuning (VeCAF). With the emerging availability of labels and natural language annotations of images through web-scale crawling or controlled generation, VeCAF makes use of these information to perform parametric data selection for PVM …

abstract arxiv collaborative cs.cv data efficiency finetuning however language novel process results tasks training type vision