Hyperbolic Learning with Synthetic Captions for Open-World Detection

arXiv:2404.05016v1 Announce Type: new
Abstract: Open-world detection poses significant challenges, as it requires the detection of any object using either object class labels or free-form texts. Existing related works often use large-scale manual annotated caption datasets for training, which are extremely expensive to collect. Instead, we propose to transfer knowledge from vision-language models (VLMs) to enrich the open-vocabulary descriptions automatically. Specifically, we bootstrap dense synthetic captions using pre-trained VLMs to provide rich descriptions on different regions in images, and incorporate …

abstract arxiv captions challenges class cs.cv datasets detection form free knowledge labels language object open-world scale synthetic training transfer type vision world