EgoLifter: Open-world 3D Segmentation for Egocentric Perception

arXiv:2403.18118v1 Announce Type: new
Abstract: In this paper we present EgoLifter, a novel system that can automatically segment scenes captured from egocentric sensors into a complete decomposition of individual 3D objects. The system is specifically designed for egocentric data where scenes contain hundreds of objects captured from natural (non-scanning) motion. EgoLifter adopts 3D Gaussians as the underlying representation of 3D scenes and objects and uses segmentation masks from the Segment Anything Model (SAM) as weak supervision to learn flexible and …

3d objects abstract arxiv cs.cv data natural novel objects open-world paper perception segment segmentation sensors type world