GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery

arXiv:2403.09974v1 Announce Type: cross
Abstract: Given unlabelled datasets containing both old and new categories, generalized category discovery (GCD) aims to accurately discover new classes while correctly classifying old classes, leveraging the class concepts learned from labeled samples. Current GCD methods only use a single visual modality of information, resulting in poor classification of visually similar classes. Though certain classes are visually confused, their text information might be distinct, motivating us to introduce text information into the GCD task. However, the …

arxiv clip cs.ai cs.cl cs.cv cs.lg discovery generalized modal multi-modal type