Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It

arXiv:2312.06420v2 Announce Type: replace
Abstract: The task of online mapping is to predict a local map using current sensor observations, e.g. from lidar and camera, without relying on a pre-built map. State-of-the-art methods are based on supervised learning and are trained predominantly using two datasets: nuScenes and Argoverse 2. However, these datasets revisit the same geographic locations across training, validation, and test sets. Specifically, over $80$% of nuScenes and $40$% of Argoverse 2 validation and test samples are less than …

arxiv cs.cv data data leakage datasets localization mapping type