Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors

arXiv:2403.14526v1 Announce Type: cross
Abstract: Precise manipulation that is generalizable across scenes and objects remains a persistent challenge in robotics. Current approaches for this task heavily depend on having a significant number of training instances to handle objects with pronounced visual and/or geometric part ambiguities. Our work explores the grounding of fine-grained part descriptors for precise manipulation in a zero-shot setting by utilizing web-trained text-to-image diffusion-based generative models. We tackle the problem by framing it as a dense semantic part …

arxiv click cs.ai cs.cv cs.ro diffusion manipulation type via visual zero-shot