GOV-NeSF: Generalizable Open-Vocabulary Neural Semantic Fields

arXiv:2404.00931v1 Announce Type: new
Abstract: Recent advancements in vision-language foundation models have significantly enhanced open-vocabulary 3D scene understanding. However, the generalizability of existing methods is constrained due to their framework designs and their reliance on 3D data. We address this limitation by introducing Generalizable Open-Vocabulary Neural Semantic Fields (GOV-NeSF), a novel approach offering a generalizable implicit representation of 3D scenes with open-vocabulary semantics. We aggregate the geometry-aware features using a cost volume, and propose a Multi-view Joint Fusion module to …

abstract arxiv cs.cv data designs fields foundation framework gov however language novel reliance semantic type understanding vision