ProTeCt: Prompt Tuning for Taxonomic Open Set Classification

arXiv:2306.02240v2 Announce Type: replace
Abstract: Visual-language foundation models, like CLIP, learn generalized representations that enable zero-shot open-set classification. Few-shot adaptation methods, based on prompt tuning, have been shown to further improve performance on downstream datasets. However, these methods do not fare well in the taxonomic open set (TOS) setting, where the classifier is asked to make predictions from label sets across different levels of semantic granularity. Frequently, they infer incorrect labels at coarser taxonomic class levels, even when the inference …

abstract arxiv classification classifier clip cs.cv datasets few-shot foundation generalized however language learn performance prompt prompt tuning protect set type visual zero-shot