Point Clouds Are Specialized Images: A Knowledge Transfer Approach for 3D Understanding

arXiv:2307.15569v2 Announce Type: replace
Abstract: Self-supervised representation learning (SSRL) has gained increasing attention in point cloud understanding, in addressing the challenges posed by 3D data scarcity and high annotation costs. This paper presents PCExpert, a novel SSRL approach that reinterprets point clouds as "specialized images". This conceptual shift allows PCExpert to leverage knowledge derived from large-scale image modality in a more direct and deeper manner, via extensively sharing the parameters with a pre-trained image encoder in a multi-way Transformer architecture. …

abstract annotation arxiv attention challenges cloud costs cs.cv data images knowledge novel paper representation representation learning shift transfer type understanding