ELiTe: Efficient Image-to-LiDAR Knowledge Transfer for Semantic Segmentation

arXiv:2405.04121v1 Announce Type: new
Abstract: Cross-modal knowledge transfer enhances point cloud representation learning in LiDAR semantic segmentation. Despite its potential, the \textit{weak teacher challenge} arises due to repetitive and non-diverse car camera images and sparse, inaccurate ground truth labels. To address this, we propose the Efficient Image-to-LiDAR Knowledge Transfer (ELiTe) paradigm. ELiTe introduces Patch-to-Point Multi-Stage Knowledge Distillation, transferring comprehensive knowledge from the Vision Foundation Model (VFM), extensively trained on diverse open-world images. This enables effective knowledge transfer to a lightweight …

abstract arxiv car challenge cloud cs.cv diverse image images knowledge labels lidar modal paradigm representation representation learning segmentation semantic transfer truth type