Unifying Scene Representation and Hand-Eye Calibration with 3D Foundation Models

arXiv:2404.11683v1 Announce Type: cross
Abstract: Representing the environment is a central challenge in robotics, and is essential for effective decision-making. Traditionally, before capturing images with a manipulator-mounted camera, users need to calibrate the camera using a specific external marker, such as a checkerboard or AprilTag. However, recent advances in computer vision have led to the development of \emph{3D foundation models}. These are large, pre-trained neural networks that can establish fast and accurate multi-view correspondences with very few images, even in …

abstract advances arxiv challenge computer cs.cv cs.ro decision environment foundation however images making representation robotics the environment type