OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network

arXiv:2209.05946v2 Announce Type: replace
Abstract: The advancement of object detection (OD) in open-vocabulary and open-world scenarios is a critical challenge in computer vision. This work introduces OmDet, a novel language-aware object detection architecture, and an innovative training mechanism that harnesses continual learning and multi-dataset vision-language pre-training. Leveraging natural language as a universal knowledge representation, OmDet accumulates a "visual vocabulary" from diverse datasets, unifying the task as a language-conditioned detection framework. Our multimodal detection network (MDN) overcomes the challenges of multi-dataset …

abstract advancement architecture arxiv challenge computer computer vision continual cs.cl cs.cv dataset detection language multimodal natural natural language network novel open-world pre-training scale training type vision work world