Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs

arXiv:2403.04114v1 Announce Type: cross
Abstract: Deep learning methods for perception are the cornerstone of many robotic systems. Despite their potential for impressive performance, obtaining real-world training data is expensive, and can be impractically difficult for some tasks. Sim-to-real transfer with domain randomization offers a potential workaround, but often requires extensive manual tuning and results in models that are brittle to distribution shift between sim and real. In this work, we introduce Composable Object Volume NeRF (COV-NeRF), an object-composable NeRF model …

abstract arxiv cs.cv cs.lg cs.ro data deep learning domain gap object perception performance randomization robotic sim systems tasks training training data transfer type visual workaround world