Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptation

arXiv:2403.11511v1 Announce Type: cross
Abstract: This paper focuses on the sim-to-real issue of RGB-D grasp detection and formulates it as a domain adaptation problem. In this case, we present a global-to-local method to address hybrid domain gaps in RGB and depth data and insufficient multi-modal feature alignment. First, a self-supervised rotation pre-training strategy is adopted to deliver robust initialization for RGB and depth networks. We then propose a global-to-local alignment pipeline with individual global domain classifiers for scene features of …

abstract alignment arxiv case cs.cv cs.ro data detection domain domain adaptation feature global hybrid issue modal multi-modal paper pre-training rgb-d rotation sim strategy training type