PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck

arXiv:2403.05297v1 Announce Type: new
Abstract: CLIP-based classifiers rely on the prompt containing a {class name} that is known to the text encoder. That is, CLIP performs poorly on new classes or the classes whose names rarely appear on the Internet (e.g., scientific names of birds). For fine-grained classification, we propose PEEB - an explainable and editable classifier to (1) express the class name into a set of pre-defined text descriptors that describe the visual parts of that class; and (2) …

abstract arxiv birds class classification classifiers clip cs.ai cs.cl cs.cv encoder fine-grained image internet language part prompt text the prompt type