DeiSAM: Segment Anything with Deictic Prompting

arXiv:2402.14123v1 Announce Type: new
Abstract: Large-scale, pre-trained neural networks have demonstrated strong capabilities in various tasks, including zero-shot image segmentation. To identify concrete objects in complex scenes, humans instinctively rely on deictic descriptions in natural language, i.e., referring to something depending on the context such as "The object that is on the desk and behind the cup.". However, deep learning approaches cannot reliably interpret such deictic representations due to their lack of reasoning capabilities in complex scenarios. To remedy this …

abstract arxiv capabilities concrete context cs.ai cs.cv cs.lg humans identify image language natural natural language networks neural networks objects prompting scale segment segment anything segmentation something tasks type zero-shot