FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

**Paper**: [https://arxiv.org/abs/2401.01970](https://arxiv.org/abs/2401.01970)

**Abstract**:

>Precisely perceiving the geometric and semantic properties of real-world 3D objects is crucial for the continued evolution of augmented reality and robotic applications. To this end, we present Foundation Model Embedded Gaussian Splatting (**FMGS**), which incorporates vision-language embeddings of foundation models into 3D Gaussian Splatting (GS). The key contribution of this work is an efficient method to reconstruct and represent 3D vision-language models. This is achieved by distilling feature maps generated from image-based foundation models into those …

3d objects abstract applications augmented reality computervision embedded embeddings evolution foundation foundation model key language objects reality robotic semantic the key vision work world