A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets

arXiv:2403.06295v1 Announce Type: new
Abstract: Few-shot Class-Incremental Learning (FSCIL) poses the challenge of retaining prior knowledge while learning from limited new data streams, all without overfitting. The rise of Vision-Language models (VLMs) has unlocked numerous applications, leveraging their existing knowledge to fine-tune on custom data. However, training the whole model is computationally prohibitive, and VLMs while being versatile in general domains still struggle with fine-grained datasets crucial for many applications. We tackle these challenges with two proposed simple modules. The …

abstract applications arxiv challenge class cs.cv data datasets data streams few-shot fine-grained however incremental knowledge language language models multimodal overfitting prior training type unlocked vision vision-language models vlms