CLIP-QDA: An Explainable Concept Bottleneck Model

arXiv:2312.00110v2 Announce Type: replace
Abstract: In this paper, we introduce an explainable algorithm designed from a multi-modal foundation model, that performs fast and explainable image classification. Drawing inspiration from CLIP-based Concept Bottleneck Models (CBMs), our method creates a latent space where each neuron is linked to a specific word. Observing that this latent space can be modeled with simple distributions, we use a Mixture of Gaussians (MoG) formalism to enhance the interpretability of this latent space. Then, we introduce CLIP-QDA, …

abstract algorithm arxiv classification clip concept cs.cv foundation foundation model image inspiration modal multi-modal neuron paper space type word