Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding

arXiv:2312.02244v2 Announce Type: replace
Abstract: Zero-shot 3D point cloud understanding can be achieved via 2D Vision-Language Models (VLMs). Existing strategies directly map Vision-Language Models from 2D pixels of rendered or captured views to 3D points, overlooking the inherent and expressible point cloud geometric structure. Geometrically similar or close regions can be exploited for bolstering point cloud understanding as they are likely to share semantic information. To this end, we introduce the first training-free aggregation technique that leverages the point cloud's …

abstract aggregation arxiv cloud cs.cv language language models map pixels strategies type understanding via vision vision-language models vlms zero-shot