InstructDET: Diversifying Referring Object Detection with Generalized Instructions

arXiv:2310.05136v5 Announce Type: replace-cross
Abstract: We propose InstructDET, a data-centric method for referring object detection (ROD) that localizes target objects based on user instructions. While deriving from referring expressions (REC), the instructions we leverage are greatly diversified to encompass common user intentions related to object detection. For one image, we produce tremendous instructions that refer to every single object and different combinations of multiple objects. Each instruction and its corresponding object bounding boxes (bbxs) constitute one training data pair. In …

abstract arxiv cs.ai cs.cv data data-centric detection generalized image object objects type