The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understanding

arXiv:2311.17518v2 Announce Type: replace-cross
Abstract: Recent advancements in large vision-language models enabled visual object detection in open-vocabulary scenarios, where object classes are defined in free-text formats during inference. In this paper, we aim to probe the state-of-the-art methods for open-vocabulary object detection to determine to what extent they understand fine-grained properties of objects and their parts. To this end, we introduce an evaluation protocol based on dynamic vocabulary generation to test whether models detect, discern, and assign the correct fine-grained …

arxiv cs.ai cs.cv cs.lg detectors fine-grained object type understanding