InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogates

arXiv:2311.03547v2 Announce Type: replace-cross
Abstract: Deep learning models are widely used in critical applications, highlighting the need for pre-deployment model understanding and improvement. Visual concept-based methods, while increasingly used for this purpose, face challenges: (1) most concepts lack interpretability, (2) existing methods require model knowledge, often unavailable at run time. Additionally, (3) there lacks a no-code method for post-understanding model improvement. Addressing these, we present InterVLS. The system facilitates model understanding by discovering text-aligned concepts, measuring their influence with model-agnostic …

abstract applications arxiv challenges concept concepts cs.ai cs.cv cs.hc cs.lg deep learning deployment face highlighting improvement interactive interpretability knowledge language replace type understanding vision vision-language visual while