CDUL: CLIP-Driven Unsupervised Learning for Multi-Label Image Classification

arXiv:2307.16634v2 Announce Type: replace
Abstract: This paper presents a CLIP-based unsupervised learning method for annotation-free multi-label image classification, including three stages: initialization, training, and inference. At the initialization stage, we take full advantage of the powerful CLIP model and propose a novel approach to extend CLIP for multi-label predictions based on global-local image-text similarity aggregation. To be more specific, we split each image into snippets and leverage CLIP to generate the similarity vector for the whole image (global) as well …

abstract annotation arxiv classification clip cs.cv free global image inference novel paper predictions stage training type unsupervised unsupervised learning