Optimization Efficient Open-World Visual Region Recognition

arXiv:2311.01373v2 Announce Type: replace
Abstract: Understanding the semantics of individual regions or patches of unconstrained images, such as open-world object detection, remains a critical yet challenging task in computer vision. Building on the success of powerful image-level vision-language (ViL) foundation models like CLIP, recent efforts have sought to harness their capabilities by either training a contrastive model from scratch with an extensive collection of region-label pairs or aligning the outputs of a detection model with image-level representations of region proposals. …

abstract arxiv building capabilities clip computer computer vision cs.ai cs.cv detection foundation harness image images language object open-world optimization recognition replace semantics success training type understanding vision vision-language visual world