Is CLIP the main roadblock for fine-grained open-world perception?

arXiv:2404.03539v1 Announce Type: new
Abstract: Modern applications increasingly demand flexible computer vision models that adapt to novel concepts not encountered during training. This necessity is pivotal in emerging domains like extended reality, robotics, and autonomous driving, which require the ability to respond to open-world stimuli. A key ingredient is the ability to identify objects based on free-form textual queries defined at inference time - a task known as open-vocabulary object detection. Multimodal backbones like CLIP are the main enabling technology …

arxiv clip cs.cv fine-grained open-world perception type world